The answer is "yes" and "no" depending on how far underground you are. The key is that all masses in the universe (every particle except light in other words) has a gravitational pull to it. The further you are from the mass, the weaker the gravitational force. Newton figured out that the relationship between two objects is their masses multiplied, divided by the distance between them squared, all multiplied by a constant number. So the further you are from something, the less gravitational force you experience. Obviously as you go into space, you feel less of a pull toward the Earth's gravity, but if you go in the other way, things get a little bit strange.
As you start digging into the Earth you'll find yourself getting closer and closer to the most massive and dense part (the outer core). So the gravity will gradually increase as you go deeper and deeper. But something else is happening at the same time. As you dig down, you end up with more of the Earth "above you". Say you've dug 12 km into the ground, you now have 12 km worth of planet above your head and that's going to have a gravitational effect, pulling you upward.
Up until you get to the outer core of the Earth, the "downward" direction wins and you will feel more and more gravity. But once you go past it into the inner core, the effect starts reversing. You find that you're now approaching a point where all the gravity is pulling you outward in all directions. At the very centre of the planet, there's nothing pulling you inward anymore, but you have an entire planet's gravity pulling you outward. The effect is that, rather surprisingly, at the centre of the Earth you become weightless.
So as you dig, Earth's gravity increases up to a point, then tails off to zero. The only problem now is surviving the insane heat of a planet's core!