I have a three year old, and go down the allotment after she has gone to bed. So I get about an 1 1/2 in a night at the mo. You don't say whether you are a single parent or not to know if this is feasible in your case.
I would check your local area and see if there is a waiting list for a plot, you may find that you have a few years wait to even get one, in which case the kids can give you a hand.