There is a gigantic literature on Bayesianism both in statistics and in philosophy. A good place to start for us are the Stanford Encyclopedia articles:
Readable philosophical books defending Bayesian views include:
In class, I was discussing how updating by conditionalization is related to the problem of induction. Not everybody thinks it is, but Colin Howson has a book that says it solves the problem and Sandy Zabell has a good paper which not only serves as a gentle introduction to exchangeability, but has a very good discussion of a great many things related to our class and is the source of basically everything I said in class about de Finetti and the problem of induction.
Please ask me if you want to find outside reading about specific topics related to probability and induction.