We did the 'Into the Glacier' tour which turned out to be the highlight of our 7 day trip to Iceland. Our guide Magdalena was very knowledgable and she kitted each of us out with a helmet, crampons and an ice pick. They also had hiking boots for hire at their office. The overall tour took about 4hrs which included a 20min hike on gravel to and from the mini-bus. All other hiking was done on the ice itself where we got to explore 3 ice caves in total. We had a lady on the tour who was really unfit and she struggled so you need to be able to handle a bit of a walk to enjoy this.
These glaciers are shrinking due to climate change so opportunities to see them will become scarce in the coming decades. Keep this in mind when booking your trip!