To me, backwards and forwards implies a next and previous, ie: past and future. I read somewhere they don't exist.
Honestly though, the path is more important than the scenery. You may need to take a job you have had once before, but you aren't the same person you were when you quit. It won't be the same job either, you will have a different outlook and a different experience.
If however, the job feels negative to you, and feels like it will hold you back in progressing on your path, find something that won't. If it doesn't resonate with you and won't pull you along to where you want to go, ditch it.