Treasury yields fell on Thursday, giving back some of the recent gains that propelled longer-dated rates to levels not seen in decades.

The 10-year Treasury yield breached a level last seen in April 2002, before easing 5 basis points to 5.243%. The 10-year influences rates on mortgage and auto loans and credit card debt. The yield on the 30-year Treasury bond also hit its highest in 24 years before dropping more than 2 basis points to 5.613%.

Yields and prices move inversely. One basis point equals 0.01%.

Jeff Kilburg, CEO of KKM Financial, said he sees the 10-year yield pulling back to around 4.5%-4.75% if the U.S. and Iran can reach a deal to end the war. "If we're going to continue to stay in Iran, then that's going to be problematic for the 10-year yield."

Loading chart...

Government borrowing costs around the world continued their relentless march upward on Thursday, continuing a monthslong trend as investors voted with their feet over a lack of political will to tackle fiscal deficits, while inflation remains stubbornly above target and leading central banks move to push interest rates higher.

Major economies face "persistently large deficits and rising interest expenses — challenges long associated with debt-distressed emerging market sovereigns," the Institute of International Finance said last week.

Japan's 10-year yield hit its highest level since the mid-1990s. Japan's debt has come under pressure from a weaker yen and rate hikes by the Bank of Japan.

The yield on the German 10-year bund, the benchmark for the euro area, topped 3.6%, the highest since 2008, before easing back. Elsewhere in Europe, the French 10-year surged 8 basis points to 4.925%, Italy's 10-year was up 10 basis points to 4.706%, while the U.K.'s 10-year yield increased 5 basis points to 5.483%.

"It's worth noting, but people get pretty worked up about those sorts of spreads regularly, so I don't think it's a crisis point necessarily, as far as Germany versus France or Italy," said Michael Schumacher, former managing director at Wells Fargo. "If you look at some countries that have very poor structural dynamics, like the UK, and I'd be concerned about that. People don't typically trade gilts versus bunds as much, but just as a barometer of, say, market concern, I think that's one to watch."

Bonds had been moving in lockstep with oil prices, which have been turbulent as the U.S.-Israel war with Iran obstructed crude exports from the Middle East. Crude oil prices were higher on Thursday, with international benchmark Brent crude futures back above $100 a barrel.

"We could see [bond] buyers come in effectively to take advantage of those yields, which would have the effect of causing them to go down, but also one of the things that has kept the volatility in those yields in the long end of the curve has been what's going on with oil, what's going on with inflation," Nomi Prins, founder of Prinsights Global, told CNBC's "Squawk Box Europe" on Thursday.

But sovereign wealth funds and central banks, among the main long-term holders of Treasury debt, are unlikely to go along, Prins said.

"We could see movement ... in Treasury yields going down if oil prices go down significantly, if there's a resolution" in the Middle East, Prins added.

— With additional reporting from CNBC's Jeff Cox