I don't believe CFC has a minimum loan amount. They have a maximum amount they'll finance however, which is 80% of what they believe the property is worth. In your scenario, you said the loan is $20k, which I assume is the selling price of the property. Given the amount, CFC may not be be as likely to lowball you but you should not be surprised if they value the land at $15k and then they will only loan you 80% of $15k ($12k). You'll have to negotiate the price of the property down or you'll have to come up with the $8k difference. Its my belief CFC (others may as well) lowballs the valuation on raw land in an effort to increase the financial stake of the buyer, which in turn lessens their risk. Again, a $20k loan is very minimal risk to them so they may not lessen the valuation.
If you have $20k equity in your home, then that's the route I would go. Its a lot easier to get a home equity loan and you'll probably get a better rate. Then just write a check for the land.