When it comes to what to collect, there are no rules. Collect what you like! It is, however, easier for many collectors to focus or to specialize.
Most collectors start by collecting worldwide stamps, or whatever is available that they can acquire. Worldwide collecting is a great way to learn about stamps and to experience the wide variety of learning available in the hobby. The stamp world has endless variety, however, and it can be overwhelming. Narrowing one’s focus makes it easier to keep a collection in order and to develop one’s expertise.
The two most common ways to collect are by geography and by topic. Many collectors focus on the stamps of one or more countries, or on the postal history of a specific place or area. It’s also possible to collect by time period (such as worldwide stamps issued during World War II). These collections have the advantage of easier access to printed albums and reference literature (such as stamp catalogs listing all of the issues of a particular country). Many countries have a few expensive or rare stamps that may be beyond the financial means of most collectors, which can be frustrating if completion is a goal.
By contrast, topical or thematic collecting uses stamps to tell a story about the subject of the stamps – whether it’s ships, butterflies, U.S. Presidents, or the space stamps shown here. Stamps can be found for virtually any subject, and some topics have literally thousands of relevant stamps. Advanced collectors add to their collections by finding cancels, postmarks and other postal-related items beyond just stamps. In topical collecting the collector sets the topic and the scope of the collection: completion is whatever the collector says it is. Here’s a nice article on topical collecting. Alternatively, you can visit the website of the American Topical Association for a wealth of information!
Endless other variations are possible: some collectors collect only specific types of stamps (such as airmail) or limit their collection to specific stamp issues (such as the United States definitive stamp issue of 1902). Anything is possible!