Isaac Prosser gave them the lead on 16 minutes before Peter Atkinson-Jones levelled past the half hour mark.
Second half goals from Sam Pinnegor, Jack Davis and Diesel Wilkes got them the win.
The result means that every Division One side has tasted defeat already this season, paving the way for a very open and competitive campaign.
Ulverston have moved up to second after the win, and Butler (pictured above) - into his fourth season at Rangers in his second spell - is pleased with their “good start”.
“We were a little bit disappointed with Tuesday (losing to Carnforth Rangers) but it’s never easy to go there”, said Butler.
“It was a great way of bouncing back yesterday, I imagine Hesketh Bank will be up there at the end of the season, it’s always a tough game.
“We played really well in the first 20 minutes, went 1-0 up and were in control of the game, we then had a water break, went back out and just conceded a sloppy goal from a corner.
“They’re quite a direct and physical side and they just started to boss us a little bit so we regrouped at half time and said we were going to have to match them in those sort of areas.
“We grew into it after the first 10 of the second half and got the three goals.
“I made changes earlier than I usually would because of the heat and they made the difference.
“Diesel Wilkes came on, set one up, scored one and he was just a nuisance!
“It was just something different that their defence couldn’t handle.”
Butler says that although he likes to “take one game at a time” with his young squad, the goal is to build on last season’s fourth placed finish.
Many of Ulverston’s players have been guided through their careers by Butler, from the Colts - which he set up a few years ago - to now.
“I try and promote young lads from around the area, my job is in youth development coaching so I do know a lot of the youngsters round here and try to bring them in.
“We are getting better, we’ve been sixth and fourth in the last two years and the lads are maturing now so I’m hoping that this year, we’ve got a better chance of maintaining the challenge and being in and around the mix come the end of the season.”
Off the pitch, there are big plans too with whispers of a possible new ground which could propel the club forward.
“My ambition is to get the team into the Premier Division but then there’s no reason why we can’t be a North West Counties side and, if we can get this new land and build a new stadium, that has to be your goal for it.
“We’ve got to focus on getting our youth setup right and attracting young players to come through each year.”
Ulverston are taking part in the first West Lancs League Groundhop next weekend, hosting Slyne with Hest at 5pm on Saturday (August 23) at North Lonsdale Road.
Butler and the team are relishing the experience.
“It’ll be a good occasion, it’s a bit of a crazy thing to be told that there may be about 300 or 400 people watching so it’ll be a different task for the players.
“Hopefully we can put on a good display for them, play some nice football and give them a good watch, it’ll be great for the club to be involved with it.”
Elsewhere in Division One, in the pick of the action across a Saturday packed full of goals, Pause United returned to the top of the table with a 9-2 win over Chipping.
Pause’s Zach Clark is, far and away, the league’s top scorer so far this season, bagging on 13 occasions and teammate Ben McMeeking on seven for the campaign.
John Sharples scored an 18-minute hat trick as Croston Sports beat Crooklands Casuals 8-0 away from home, while Kendal United were 6-1 winners over Freckleton and Slyne with Hest picked up their first three points of the season as they saw off Holker Old Boys Reserves 4-1.
Four first half goals took Lostock St Gerards to the summit of the Premier Division - Mason McKay (2), John-Jo Morris and Reece Thompson were on the scoresheet before the break in a 4-0 win at Cleator Moor Celtic.
Haslingden St Marys and Poulton recorded 5-1 wins away from home at Blackpool Wren Rovers and CMB respectively, while late Eagley goals rescued a point for them in a 3-3 draw at Coppull United.
Third placed Millom, along with LSG, Eagley and Poulton remain unbeaten after a 2-1 win at Burscough Richmond.
There were an astonishing 68 goals across eight Division Two games.
Fulwood Amateurs Reserves are scoring goals for fun - 35 in their opening five to be precise - and 10 those came against Walney Island this weekend.
Charlie Howarth scored four as they earned a 10-1 victory, but that was just one of three games won by a whopping nine-goal margin.
Second placed Stoneclough and a Langho side who’ve started very solidly in the WLFL both secured 9-0 wins over Thornton Cleveleys Reserves and Galgate respectively.
The two sides also had hat trick heroes, namely Callum Darcy and Kevin Myftaraja and both added another after that.
Bamber Bridge Reserves defeated Appley Bridge 8-3 with Taylor Maguire scoring three, while Euxton Villa Reserves and Mill Hill bagged 6-3 away wins over Dalton United and Caton United.
Storeys came out on top in an eight-goal affair, leaving Charnock Richard with a 5-3 win over their Reserves.
Carnforth Rangers Reserves continue to lead Reserve Division One after a 2-0 win at Turton Reserves, while Poulton Reserves secured a nail-biting 6-4 win versus CMB Reserves and Tempest United Reserves came away 5-1 victors at Chipping Reserves.
In Reserve Division Two, Astley and Buckshaw Reserves saw off Pause Reserves 5-2, while Fulwood Amateurs A and top of the table Thornton Cleveleys A edged past Storeys and Croston Sports’ reserves.
Keep up-to-date with our exclusive email newsletters.
Subscribe