Unfortunately, your biggest mistake was not having a lease agreement. Most of the tenants, at the time of taking an apartment for rent make this mistake of either not entering into an agreement or not taking their terms into the agreement. In your case, you dont have an agreement in the first place, which would have been a proof that you have given him a security deposit. In the second place, why were you working to restore the building? Was it part of the agreement? Did your landlord agree to pay you in the first place? And do you have any written record for it? Well, if he asked you to work on it and agreed on a pay, and you have it in writing, you are safe and you can make a claim, however, in the other case, you would have to suffer. However, i suggest talk to your lawyer about it.

 Yes, he have the right but be sure to ask for the receipts.