Just because I get angry with what the Bible says does not mean that I take it any less seriously. In fact, getting angry with what the Bible has to say is an indication that I do take it to heart. This is along a similar vein to the meaning of no fights in a relationship. If I never disagree with someone nor get upset with that person then I am not sufficiently listening nor letting the other person affect me. In such a situation we are not close. The lack of conflict is a sign of a lack of true interaction.
So when my responses reach beyond either standard response of going along with what the Bible says or ignoring it while pretending to follow it then you will know that this means the Bible matters to me. Through the Bible I connect to the beliefs and experiences of Christians and Jews across time. The Bible helps to connect me with the Communion of the Saints and the Great Cloud of Witnesses.
Yes, I care about context–both the other passages around a pericope and the culture of the writer(s) and editors. I am eager to learn more about the people who created a text. Sometimes what a passage can tell me about the people who wrote it is more significant than the words on the page. Yet, I also care about the interpretations of other readers of the same text. By reading together with people throughout time I find connections.
The Bible comes out in my conceptual framework and in phrases that I use. If you are well versed in the Bible then you will be able to find the origins of many of my thoughts and turns-of-phrase there long after I have forgotten where they came from. You can scoff at me for not being Bible compliant, but I am Bible soaked.