This term is most often used to denote banks and firms that would substantially damage the financial system and the rest of the economy should they “fail”, i.e., go bankrupt.
The logic is therefore that these organisations would receive a “bailout” of some kind from the government - at the very least, protecting creditors against losses and perhaps also enabling management to stay in place (and, in some cases, the full payment of wages and bonuses).
The term has been used in the United States since at least the 1980s, when it arose in the context of the conservatorship extended to Continental Illinois.
The definitive book on the history and analytics of the topic is "Too Big To Fail", by Gary Stern and Ron Feldman (Brookings Institution Press), written in the early 2000s. Andrew Ross Sorkin’s more recent book with the same title provides a blow-by-blow account of how big US banks were rescued in fall 2008 after the collapse of Lehman Brothers.
The Dodd-Frank legislation of 2010 was argued, by its proponents, to have “ended too big to fail”, for example, by extending FDIC-style resolution to all financial institutions - even the largest and most dangerous.