In the real world, it all boils down to the type of project and the timeline associated with it. If the project in question needs to be accomplished quickly, a framework will help you get the job done by offering you a solid code base to begin with. Similarly, if the project in question is a large and complex application, a framework will offer you the bug fixes and helpful documentation that you might be in need of.
Alternatively, if you are stuck somewhere in the middle of the battle, wherein you wish to get rid of frameworks because you do not need all the complex features and functions, but cannot or do not want to code everything from scratch, choosing a nimble and lightweight framework might be of use for it. Options such as Sammy and Min.js are worthy choices that you should look at.