I hope you meant “pulled out” of the stock market, and NOT out of your SEP-IRA!
If you are just changing asset allocation inside your plan, you are trying to “time” the market. And odds are, you will be wrong! My advice is always to keep a set and appropriate percentage (which can change as you get closer to retirement) and then sleep well and let the stock market do its work over the long run!
Every time I break this rule myself, I regret it!!