I know that several independents in the US Senate (ie. Bernie Sanders) caucus with the Democrats. Officially, what does that mean? How do independents choose whom to caucus with (both ideologically and technically)? How is that different from them joining the party in question?
To understand what the Congressional Caucus is it helps to understand the history behind this word that has its roots in early American history. John Adams was the first to use the term to describe a meeting of politicians in his diary:
This day learned that the Caucas Clubb meets at certain Times in the Garret of Tom Daws, the Adjutant of the Boston Regiment. He has a large House, and he has a moveable Partition in his Garrett, which he takes down and the whole Clubb meets in one Room. There they smoke tobacco till you cannot see from one End of the Garrett to the other. There they drink Phlip I suppose, and there they choose a Moderator, who puts Questions to the Vote regularly, and select Men, Assessors, Collectors, Wardens, Fire Wards, and Representatives are Regularly chosen before they are chosen in the Town
The word in Latin literally means "drinking vessel", and the Caucas Club was a popular pub in Boston at the time. In other words, he was more or less describing them as a bunch of political figures who all happen to also be drinking buddies.
Fast forward to today in United States politics and the term has a very similar meaning, a group of politicians that are generally friendly and agree with each other on most issues. Your caucus is the group where you are most likely to find political allies with when garnering support for a pet project, pork barrel spending or a bill that you feel very strongly for.
While the two dominant parties are currently Republicans and Democrats in US politics, the caucus typically expands to include other independents or members of third parties that would most likely share the most values with your party.
The caucus lines are typically demarcated between Conservative Right and Liberal Left with independents and members of third parties making the declaration of which side they choose either way. This declaration is important on Capitol Hill as it helps other members of congress to more accurately predict the outcome of any one bill before it goes to vote. This allows the Congress to save a great deal of time and be more efficient by not wasting time voting on bills that likely have no chance of ever being passed.
Without independents declaring a caucus, they become a toss up and this reduces the efficiency of the US legislative process.
To answer what makes them different from the party themselves, it becomes evident once we get an understanding of Congressional rules. As per typical parliamentary style politics, the Congressional rules are formed with the assumption that there will always be two diametrically opposing sides, the Majority, and the Minority. Being in the Majority party grants the privilege of certain congressional appointments as well as other things. In a multi party system the many parties generally caucus together to make appointments as a majority. It is similar to the US caucus just on a much smaller scale as third parties and independents are an exception.
This is why they will form a Democratic Caucus and Republican Caucus by name, however these are not necessarily affiliated with any one political party. An independent Bernie Sanders has left leaning political ideals and thus has more in common with the Democratic party than the Republican party so he chooses to caucus with the Democrats. Another independent Joe Lieberman has been known to switch caucuses or at least entertain the notion from time to time, starting out caucusing with the Democrats then later in his political career caucusing with the Republicans.
Sanders however doesn't get nominated by the Democratic party nor do Democrats have an opportunity to vote for his nomination. Likewise the same could be said for Lieberman, he could caucus with the Republicans though Republicans in his district had no vote to nominate him.
For what it's worth, members of third parties (and independents) did not start (formally) caucusing with one of the two major parties until 1971. James Buckley was elected as a member of the Conservative Party of New York State and joined the Republican caucus for the 92nd Congress. Harry Byrd was also elected in that cycle despite breaking away from the Democratic party and running as an Independent. He subsequently rejoined the Democratic caucus, though.