I understand where you are coming from, I have had the same sort of thinking as you, although I'm not as a hard on myself lately.
I think it has to do with self esteem, anxiety and perfectionism and finding ways to prove to ourselves that what we believe about ourselves is actually the case, by picking out magnified flaws, like being bad at maths, English, history, common knowledge etc.
I know I have built up so much anxiety over the years about not being able to write properly, that I really believed I had some sort of learning difficulty (oh and I have requested IQ tests, learning difficulty tests and all came back negative). Same with maths, I dreaded ever going for basic jobs that requiring maths, such as working in a cafe because I would be too slow to do calculations with customers money.
I think my anxiety is warranted to some extent, although I am coming to realise that a lot of people around me are not that much better at English and maths and they seem to be doing fine. Even though friends and family always said they went through the same thing or that they are not that great at this and that. It's just that I care and take it personally and being a perfectionist I take it really personally that I can't grasp a concept and so I become overly anxious, which then heightens the problem.
I think anxiety can worsen when your in an environment that forces you to work with your fears. At the moment the jobs I hold don't really require me to write or do maths. That's because I know I would never be "great" in these areas, so I've tried to pave a career that doesn't require these skills as much. It still hard though, because almost all jobs need some sort of English and I freak out at being caught out with terrible grammar and all that jazz.
I have a terrible memory too. I find it takes me a long time to remember sequences or words. You can tell me a name 20 times and I still wont remember it, it actually is quite stressful, because you get stressed that you can't remember something so simple and think what's wrong with me? I have a feeling it's all to do with anxiety, so maybe that's the case with you too.
I think what would make it easier for me, was if I was just really great at one thing. Like I had this skill that just came so naturally to me.
But I am slowly learning, that you don't have to be to perfect at everything, in fact I think I am great at nothing, but I still manage to survive
By the way I work in childcare a few days a week as well. Normally I am a web designer, but I am not the best of the best, and I can be quite slow and thought I needed to be in a more, how do I put it, "less technical job". However, I have found childcare a little bit too "un-challenging" for me, surprisingly. I do find that I like to nut things out, which web design offers me, but I have to do it on my own terms, which means freelancing. I would be deemed too slow in most design firms, so freelancing allows me to go at my own pace. But to be fair, if I think about a lot of companies acspect you to know so much and be so fast without taking the time to nurture their employees, which I think it quite unrealistic and that would make a lot of people feel quite inadequate (basically having to live up to standards that are not realistic).
Most of what you have said resonates with me too, the lack of knowledge, maths, English, being able to articulate my thoughts properly in conversations ect. And you know all of these things have gotten worse for me over time because I allowed myself to believe they were true. So if I could tell someone else in my shoes on how to get better, I would say go see a psychologist and talk over your fears, get it out on the table to how your thoughts are effecting you, or what you believe to be true. A good psychologist will help reason with your thoughts.
I can tell from your post that your not dumb by any means, you just believing that to be so at this point of time. Get some help, so you don't have to travel the same journey as me. Where I ended up realising I'm not so dumb and I wasted a lot of time believing so.
I'm crap at maps and locations too, but who cares? So what you wont need it unless you want to be a taxi driver

Politcs bores me, so I don't know a thing about it. Maybe you just need to start accepting that you aren't good at this and that and that's the way the way it has to be?