Go to your Applications folder and find the Play Store, It looks likea white shopping bag with the little Play Arrow you would see on a DVD player and such. When my phone went from the "Market" to the "Play Store" my icon disappeared off the home page and was hidden within the Applications.
AND, by the way, Google Play Store is NOT the only place to get legitamite apps..... there's Amazon App Store and Getjar, too. There may be others as well. Google Play Store tries to charge $.99 for the Amazon App Store app; it's free from Amazon.