When a bank, brokerage or other financial institution trades on its own account rather than on behalf of a customer. 
In simple terms, proprietary or prop trading is where a trading desk, using the bank's own capital and balance sheet, carries out trades in various instruments, often for speculative purposes.
They can be ordinary shares and bonds traded on exchanges, but are more often derivatives - either exchange-traded or in the over-the-counter markets - or foreign exchange.
So-called "pure" proprietary trading is where traders trade for the bank's own profit, unrelated to client business. These traders are generally "walled off" from the rest of the bank, and generate only a portion of total trading revenues.
The other type of trading banks do is to help clients carry out trades, where the desk will use the bank's own capital to make a market in a certain instrument, offering itself as a buyer to a client that wants to sell, or a seller to a client that wants to buy.
Known as "flow business", this is not speculative trading by the banks. Yet, to a certain extent, it still puts the bank's own capital at risk, sometimes in as significant a way as if it were conducting its own speculative prop trading.