Can You Give A Tea Cup Chihuahua Milk?

Not human milk ever (causes severe stomach upset and doesn't have the proper nutrition)and there shouldn't ever be any need for formula after weaning.
It is also a concern, including about your pup's health that you mention the word teacup. There is no such thing in any breed and as far as chihuahua's go there is a long coat chihuahua and a smooth coat chihuahua. There are bigger or smaller ones but they are not called by anything else. A 'breeder' who uses those terms either knows this and is probably not particularly ethical for using that term for sales or is unknowledable about the breed. Neither bodes well for any buyer (or the pups produced) from several aspects. Extremely tiny pups often have more problems (ALOT of liver shunts lately due to people just breeding for small and also not testing smaller dogs being bred among other things), shorter life spans, and require additional care. Of course, 'teacup' also varies from person to person.
Here's some information on chihuahua's specifically for you.
www.chihuahuaclubofamerica.com
www.chihuahuaclubofamerica.com
www.chihuahuaclubofamerica.com

Not human milk like from cows. The only milk a puppy can drink is from it;s mother or specific milk that your dogs vet will give you.