We often hear pundits and the president, talking about bipartisanship as though bipartisanship is an inherently good thing and as though the recent decline in bipartisanship is an indicator of some kind of breakdown of the political system.
There has in-fact been a quantifiable decline in bipartisanship over the past 20 years when compared to the prior 50 years, but the reality is that the bipartisanship of the period from the 1930s through the 1980s was an anomaly caused by the process of ideological re-alignment of the two parties.
The image below shows the degree of difference between the two parties based on votes over time.
This next image shows senate voting patterns historically going back to 1857. Follow the link below for more a more detailed look:
(Note: I don't agree with the definitions of "left" and "right" in the linked material, its far too simplistic)
People often confuse partisanship with ideology, but these two things are not the same. Partisanship refers specifically to party affiliation, whereas ideology refers to one's political beliefs and motivations. It makes sense for party affiliation to be aligned with ideology, and historically it has been. However, there was a period from the 1930s through the 1980s when there was a high degree of ideological diversity within the political parties, not for any noble reasons and not because there was some kind of growing political consensus taking shape, but because of ideological re-alignments taking place within the parties, largely based on regional shifts in the North and South.
This began, essentially with the rise of wage laborers as a political force and came to prominence with the presidency of Teddy Roosevelt in the early 1900s. Historically American politics has been ideologically divided into two main groups, largely because of the de-facto two party system. That breakdown from the time of the founding up to the beginning of the 20th century was effectively socially conservative economic populism (dominated by farming interests) vs socially progressive economic elitism (dominated by northern merchants and industrialists).
With the rise of industrialization an new class of propertyless worker became significant in numbers, largely in the cities, which were dominated by the Republicans. While the Republicans had traditionally dominated the metropolitan vote, the rise of "wage-laborering" city dwellers gave rise to the need to adopt some level of populist positions to go after that vote. The vote was then split between the interests of farmers, wage laborers, and non-farm business owners. Prior to the last 19th century there were essentially only farmers and non-farm businesses owners, there was no significant population of "wage-laborers", and the rise of wage-laborers in the cities, along with the decline of the number of farmers, setup a battle for this new vote. This created ideological conflict within the Republican party, because now the metropolitan vote wasn't just dominated by the business owner's vote, now the business owners had competition from the wage laborers.
This is what gave rise to the so-called "progressive" movement, the rise of socially progressive economic populism.
Teddy Roosevelt was the first major Republican political figure to come to the side of the wage-laborers and to be a socially progressive economic populist. This began fracturing the Republican party ideologically.
The same thing was happening on the other side as well. The Democrats had traditionally been the economically populist "farmer's party", but with the rise of wage-laborers in the cities, the traditional economic populism of the Democrats found support among the wage-laborers. In addition, many wage-laborers in the cities had recently moved to the cities from farms and came from traditionally Democratic families.
This is what gave rise to the socially progressive metropolitan Northern Democrats, from which FDR sprang. These figures were also socially progressive and economically populist, likewise going after that urban wage-labor vote.
Prior to the Great Depression, however, the parties remained relatively ideologically pure, the Democrats dominating the South and the Republicans dominating the North. But with the Great Depression the solid economic populism of the Democratic party trumped everything, and so blacks from the South who had traditionally voted Republican now voted Democrat out of economic interests, and likewise wage-laborers and small businessmen from the North who had traditionally voted Republican also now voted Democrat out of economic interests. Essentially, everyone who wasn't wealthy, regardless of whether they were socially conservative or socially progressive, voted Democrat.
This mixed the social conservatives and social progressives up into the same party. Now the Democratic Party was a "big tent" on social issues with progressive Northern Democrats voting along with conservative Southern Democrats. However, because FDR was himself a progressive Northern Democrat and because he was in office for 12 years he had a huge impact on the national direction of the party, and it was FDR who sowed the seeds of the Democratic party's transformation from a Southern conservative party to a national progressive party.
From the 1930s through 1970s that transformation was taking place, and as that transformation took place a counter transformation took place on the Republican side, with the Republican party transforming from a Northern progressive party to a Southern conservative party. But both parties retained their ideological roots on economic issues, the Democrats being populists and the Republcians being the party of "big business".
The bipartisanship of the 1930s-1980s was really just a byproduct of this realignment. Due to the fact that it took a few generations for traditionally conservative Democrats to die out or leave the party, what we had from the 1930s to the 1980s were vestiges of the Democrat's conservative past lingering in the party. The same goes on the Republican side, with vestiges of the Republican's progressive past lingering on.
These people still voted along ideological lines. The ideological lines remained just as heavily drawn the whole time, the only difference was that the composition of the parties was changing ideologically.
This is where the difference between cross-party agreement and cross ideological agreement becomes important. We talk about bipartisanship because there is an implication that if legislation can get bipartisan support it must mean that it's not controversial, because "everyone" can agree that it's good. But we can't really compare the bipartisanship of the 1930s-1980s with today, because the parties back then were much more ideologically diverse. What really tells you if legislation is "controversial" or not isn't whether it gets bipartisan support, but whether it has support across the ideological spectrum, and the truth is that there was just as much division ideologically from the 1930s through to the 1980s as there is now, it's just that the parties weren't as ideologically distinct, because of the period of ideological realignment.
So what does all this mean? It means that going on and on about the bipartisanship of the past is at best largely meaningless, and is generally mis-representative Talking about the bipartisanship of the past implies that there weren't ideological conflicts in the past, which is patently false, there were, and progressive policies had to be fought for along ideological lines. It just so happens that those ideological lines weren't the same as the party lines. There was less conflict between the parties, but there was more conflict within them.
On an additional note, while partisanship is arguably increasing today compared to the past 50 years, both parties could actually be considered more ideologically similar today than they were in the past as well, due largely to the changing balance of power in the American political system. As wealth has become more concentrated among the super-rich, the political power has become more concentrated as well. As a result, both parties are chasing after the same base of power, the interests of the super-rich and the corporations. A major distinction has to be noted. Prior to "The Great Realignment" from the 1930s-1970s American economic populism was rooted in the political power of the farmers. Now, however, having come out of the realignment, that power is gone, there is no longer any base of independent farmers which is politically powerful, since today only 2% of the population are farmers, and most farms are now run by large corporations, making them more like normal corporations than traditional farms in terms of their interests. In addition, the rise of retired seniors has changed the political landscape. The interests of working-class populism now resides among wage-laborers, but wage-laborers are weaker politically than the farmers were, and retired seniors have conflicting interests with working wage-laborers. So unfortunately the forces of economic populism, while still dominate in terms of numbers in America, are politically much weaker. So, despite the return of partisanship, the reality is that the differences between the two parties are smaller today than in the pre-realignment past, with greater fighting over less significant differences and with the differences being more pronounced on less economically impactful social issues.