The cornerstone of thermodynamics is the Carnot cycle, an ideal process for converting heat to work. It's a model for what steam engines do, for example. The Carnot cycle sets a limit on how efficient an engine can be: it is not possible to convert all the energy from heat to mechanical work.
A heat pump is simply an engine running backwards. An engine has heat flowing from a hot reservoir to a cold reservoir, converting some of that heat to mechanical work. A heat pump uses mechanical work to push heat from a cold reservoir to a hot reservoir. The amount of heat added to the hot reservoir will be the sum of the energy from the work and the heat energy removed from the cold reservoir.
To heat a home, one can use a natural gas furnace, or one can use a heat pump. The heat pump runs off electricity, much of which is generated from an engine running off natural gas. Energy is lost when the natural gas heat energy is converted to electricity, but then energy is gained when the electricity is used to heat the home. Since the heat pump is just an engine running backwards, these losses and gains are in some sense reflections of each other, and might seem to cancel out. But they don't!
The missing detail is that there are three heat reservoirs involved. The engine at the utility power generation plant has energy flowing from a furnace to the environment, converting some of that to electrical energy. The heat pump has energy flowing from the environment to the interior living space, driving that with electrical energy:
The two efficiency factors have inverse forms, but the numbers involved are different, so they don't cancel each other.
Plugging in some roughly plausible numbers, a graph can be generated for maximum effiency of the overall system as a function of the outside temperature. As the outside temperature warms to near the interior living space temperature, the round trip efficiency increases without bound. At cold temperatures, the utility's power generation engine can run more efficiently, but the reduction in effectiveness of the heat pump is more dramatic, so the overall effiency is reduced.