It's probably fine. Something you might want to consider is what's called a "play yard" from Petco or Petsmart and set it up in your house. This way he has room to play with his toys while you are gone.