
The first thing you should consider when you start a stamp collections is this: start simple. As I said, the common stamps are cheap and you should start with them until you learn how to handle and how to preserve them, until you have a solid base to start expanding from. In a lot of places in the world, where stamp collecting has a history, you can find affordable packets of used stamps at stamp dealers and hobby stores. You can also find the currently used stamps in the postal offices in your country, there are also targeted websites to this field (you can buy stamps online). Make sure that the beginner's pack of stamps contains all different types as variety and quantity.
The next step is to sort your collection. Sorting by country (it is desired to start your collection with stamps from different countries) is one of the most common methods. Another methods is the sort by the subject represented on the stamp or the historical period.
In order to better understand the way will take with your collection find more about the type of the stamps existing in the world you should research for information. There are still catalogues of stamps printed on paper, you can always order some, but searching the internet is easier: join an online discussion group or a stamp collectors group to gain more knowlege about this field.
The next step is to identify the stamps you are not interested in (as categories and items in your existing collection). You can always find other stamp collecting enthusiasts to swap stamps or to sell them to, any option can be beneficial and/or fun for you.
As your collection grows you can get a good idea about the values of your stamps and how rare they are. Use the catalogues (on paper or online) and price guide for any given stamp issue. All these may decide the direction of your collecting interests.
As a final thing you need to start a stamp collection is the list of items for the beginner's outfit. There are a lot of outfits on the market, especially if you know how to search them, but essentials are: a stamp album, stamp hinges, stamp tongs and a packet of 1000-2000 stamps of the world. Any stamp dealer or any experienced stamp collector can help you with these.