Where does the belief the Bible is written by the Holy Spirit come from?

My pastor mentioned the Bible is written by the Holy Spirit, through man. Where does this concept come from (rather than just written by wise or experienced men)? Other than direct references to Jesus, how do we know the rest is not just man's ideas and are indeed God inspired or "written"? Is there a chapter in the Bible that states this or is it tradition?