The difference between Roomba 770 and Roomba 870 are as follows.
Big Ones: Cleaning System, Filtration, Bin.

Roomba 870 vs 770
images from manufacturer

Cleaning System
They use different cleaning system.
The 770 uses AeroVac Series 2 Cleaning System whereas the 870 uses the AeroForce Performance Cleaning System which was claimed to be more powerful and requires less maintenance.
As part of the cleaning system, the 870 uses 2 brushless rubbery extractors to replace 1 bristle brush and 1 rubber brush on the 770.

Filtration
Both of them use HEPA filtration, however, they are different. The 770 uses Dual HEPA Air Filters that are smaller whereas the 870 uses single larger HEPA filter.

Bin
The sealed, HEPA-filtered bin on the 870 is larger than the AeroVac Series 2 Bin on the 770. According to user reviews, it is easier to clean the 870 bin.

Price
There is a $100 difference in price.
The Roomba 770 is $100 cheaper than the Roomba 870 which is price at $599.99 at irobot.com.

Home Base
The 770 uses a Battery Charger and a Compact Self-Charging Home Base whereas the 870 uses the Integrated Home Base.

Battery
The 770 uses the Advanced Power System Battery whereas the 870 uses the XLife Extended Life Battery that was claimed to have longer lifespan.

Accessories
The 770 comes with a Remote Control that allows you to control Roomba without getting up whereas the 870 does not include.
Both of them comes with 2 virtual walls, however, the 870 includes C batteries for the virtual walls whereas the 770 does not include.
The 770 includes 2 brush cleaning tools for removing hair and debris from the bristle brush that the 870 does not include.

Similarities:
Features Full Bin Indicator
on-board scheduling,
self docking and charging,
Dirt Detect Series 2 with Persistent Pass Cleaning Method,
weighs 8.4 lbs, 13.9 inches in diameter, 3.6 inches in height,
no room to room navigation.

If you know more difference between Roomba 770 or 870, please share below, thank you.