In the original Star Wars trilogy, there are like 5 people who use the Force total. That's it. Sure, there may have been more people who were Force sensitive but didn't use it (like Leia), but it's a pretty huge minority. Most people are just regular, non-Force-having, normal folks - and that's probably how it would be if the Force were a real thing. The odds of you having it are slim to nil - and the odds of you actually being Force sensitive enough to actually DO anything with it is even slimmer. Not everyone can be Skippy the Force-Droid, after all.
The Force can be used for lots of good in the world - or, more likely, people just being total dicks to one another. The problem with introducing that kind of unfathomable power to the world is that it works on the honor system - there's no way to really regulate its use. So Force-users could be running around choking people to death with SPACE MAGIC and never be caught. How would our legal system be able to adapt to a "well, he probably has magic powers" as a prosecution angle?
Hey guess what - Lightsabers have jackshit to do with The Force. It's just a totally separate thing altogether - the Force is a mystical energy that binds together all living things, and lightsabers are just cool laser-swords that were invented by some dude. But since lightsabers are one of the coolest things about being a Jedi or Sith, you'd kinda assume you'd naturally get to have one. Nope - even if space magic were real, laserswords are still going to be an unrealized dream.
Let's face it - nearly everyone who had The Force would use it for personal gain or straight-up evil. Sure, maybe people would do nice things every now and then, but more often than not they'll use it for petty revenge or massive power plays. A well-trained Force user would basically be an IRL wizard, able to fell entire armies with the flick of a wrist. Nations would fall, the economy would be disrupted, millions would perish, all because a few bad apples got powers.
And - perhaps even more terrible to imagine - many might become street magicians.