I made apple and mint jelly last week. The mint was looking scraggly and old and the stalks were bvery tough but the leaves are normally fine. The only issue is it takes longer to pick enough leaves!!!
I think the best time to pick is when they are very young, you can use the whole thing then