Hundreds of their supporters are trying to show that British fans have a big heart by bringing aid to some of the most disadvantaged people in this poor country on the fringes of Europe.
The idea arose on a previous away trip to Finland, as fan Neil Dymock explains: "We decided we could do more than just watch the game - when we got back, we got in touch with some charities in Baku, the capital of Azerbaijan, and asked how we could help.
"It's a way of giving something back to the people who make our away trips such an adventure!"
The fans have been overwhelmed by offers of help - clubs and schools have given kit and footballs - and donations have arrived from as far away as the USA.
One of the groups to benefit are the childen at the "Number One" orphanage in Baku.
Many lost their parents in a war against Armenia.
Others have parents who are simply too poor to look after them.
Some of the children are doing "sports therapy" so the offer of football kit is of real value.
Farid Talebov, director of United Aid for Azerbaijan, which helps the orphanage, told BBC 5 Live: "Some British fans are famous in Azerbaijan for the wrong reasons.
"So we were amazed - but delighted - when the Welsh supporters offered to help us."
At "Welsh Night" in the Lancaster Gate pub in Baku fans not only sang their hearts out - they also raised hundreds of pounds for the charity Lotos, which works to integrate disabled people into Azeri society.
Travelling with the red-shirted Welsh army is a real experience - if anyone can give British soccer a positive image abroad, its Welsh fans.
Next stop for the Welsh team is Yugoslavia, and fans are already starting to collect to help orphanages there.
All that remains now is for Wales to get the points on the board and send the fans home doubly happy.