What kind of content should one avoid when publishing (math) notes online?
Copyright protects published texts and images, not the underlying ideas. A concept such as a "Fourier transform" cannot be copyrighted, no matter how it's written or described. However, what you cannot do is republish the material that's been put under copyright without obtaining permission, or demonstrating a clear fair-use case. For instance, one or two short, relevant quotations that are properly cited in a 50-page manuscript will not get you into trouble. Reprinting larger amounts or more complicated material (tables, graphics, etc.), however, should be done via a copyright request to the publisher.

You cannot reprint the text of problems that have been published elsewhere (unless, of course, it's sufficiently generic that it could be found in multiple books. For instance, the following text would not likely be copyrightable:


  Prove that a connected open subset of R^n is pathwise connected. 


for the reason that there is no "unique" or "original" content here; literally any book in the field could pose the same problem in exactly the same way. However, a longer, more detailed statement of the problem, guiding you through specific steps, would probably not be publishable.

On the other hand, the solution of such problems, so long as they represent independent scholarly work, need not be "suppressed." Again, copyright applies to the authors' wording of the solutions of the problems; the method for solving them cannot be copyrighted! If you solve them yourself, and publish your own solutions, that's not violating copyright. (There is the issue of making the lives of future students easier, because you're publishing solutions online that they could plagiarize from, but that's a separate issue from if you're allowed to publish such material.)