TCP Getting started guide

This guide is meant to be your first resource for getting started with the TCP project after working on IP. We’ll be adding more implementation resources in the next couple of days. For now, your work should be mainly focused on design.

Essential resources

As you work on the project, you should be aware of the following resources for each stage of the project.

Milestone 1

  • Lecture 12: gives an overview of how sockets work and describe the TCP handshake, the most important components for starting to think about and work with sockets. Lecture 11 also discusses the fundamentals of thinking about sockets. You should watch these before you start–it will help, we promise!
  • TCP Gearup I (notes, recording) gives an overview of the project, how to try out things using the reference, and what you need for milestone 1–particularly how to think about creating sockets
  • The TCP-in-IP Example is a complete code example for how to build and parse TCP packets and encapsulate them in virtual IP packets. This will be helpful for understanding how to serialize headers into byte arrays, and how to integrate this process into what you already have from IP.
  • The Channels example demonstrates one way to use channels to signal another thread to handle an event, which you may find useful for waiting for packet events

Beyond milestone 1

Milestone 2

  • Lecture 14: discusses the basics of how sending and receiving should work
  • Lecture 15: provides details on how sliding window works. See the notes for diagrams!
  • TCP Gearup II (notes, recording): how your send (s) and receive (r) commands should work, more sliding window examples, how to signal threads to send/receive data
  • The channels demo demonstrates one way to use Go channels to signal threads
  • :sparkles: HW3 :sparkles: will be a warmup for Milestone 2–do this before you work on Milestone 2, it will help!

Final deadline

  • Lecture 15: talks about retransmissions, early arrivals (out-of-order packets), and zero-window probing
  • Lecture 16 (at the beginning): talks about connection termination
  • TCP Gearup III (notes, recording) demonstrates all of these features and how to test them in wireshark (Note: recording is from last year due to a recording issue)

Creating your repository

To create your repo, one team member MUST follow the instructions here to create your initial repo and migrate your git history from IP. If these steps have already been completed, you can clone your repo from your new URL as normal and jump to Running TCP.

If you are changing teams, you should (as a team) choose which of your IP repositories to use as a starting point for TCP. Once you decide, the team member that has access to the repo you’ll be using should do these steps.

If you are the team member setting up the repository, do NOT skip this part, and don’t improvise: Since you’ll be creating a new repository based on your IP repo, the procedure for creating your repo is different than a normal stencil, so DO :clap: NOT :clap: SKIP :clap: STEPS :clap:.

If you don’t follow the instructions, you’ll likely run into errors creating your repo, or your file permissions will be incorrect, which will cause problems later!

To create your repository, do the following:

  1. Visit the Github classroom link in the email you received. If you did not receive an email, please check Edstem for the latest updates/instructions.

  2. If your team members have not yet created a TCP team, create a new team, as shown on the figure below. Even if you are staying on the same team, you MUST pick a new team name–this is an unfortunate limitation of Github classroom. To pick a new team name, you can simply add “tcp” to your team name (“team-99”=> “tcp-team-99”), or pick something fun (“team 99 strikes back”)

    If you see the following (very misleading) error:

    The team name you have picked already exists! Please pick a new one.

  3. Once your team has been created, accept the assignment and proceed to your repository view. The repository you will have just cloned is completely blank, so your repo page will look like this:

Migrating your IP code

Next, we need to push your IP code to this new repo. To do this:

  1. As shown in the figure, copy the git URL for your new repo. Use whichever URL type your normally use to clone stencils (HTTP or SSH).

  2. In whatever terminal you normally use for git, cd to your IP repo (if you are changing teams, “you” should be the owner of the IP repo your team will be using for TCP).

  3. In your IP repo, enter the following commands

$ git remote add tcp <URL you copied from your TCP repo>
$ git push tcp main
[ . . .]
To github.com:brown-cs1680-f24/<new repo URL>.git
 * [new branch]      main -> main

This adds your TCP repo as a new “remote” to your IP repo, which is a location where you can push your code. If all goes well, your output should look like the example.

If you run into access errors, make sure you’re using the correct URL type (eg. run git remote -v and use the same type listed for your IP URL (named origin)).

  1. To make sure the push was successful, refresh your TCP repo on Github–you should now see all your files from IP, like this:

This means the migration was successful! You’re almost done! Continue on to update your repo for TCP.

Cloning and updating your repo for TCP

Now that your IP code has been migrated to your new repo, you’ll need to clone your new TCP repo to a new directory and pull in an updated version of our reference that implements TCP. To do this:

  1. If you have not done so already, clone your new TCP repo just like any normal repo. As with IP, we recommend cloning your repo in the DEV-ENVIRONMENT/home directory, like this:
 - ...
 |--DEV-ENVIRONMENT
 | |--docker/ 
 | |--home/
 | | |--snowcast-yourname/
 | | |--ip-team-name/
 | | |--tcp-new-team-name/     # <----------------- Clone your repo here!
 | |--run-container
 | |-- ...
 ...
  1. Once you’ve cloned the repo, open a container terminal inside your new TCP repo
  2. Run the script util/update_from_stencil. This should automatically pull the changes from our template repo. If the update is successful, you should see several files update, which should look similar to this:
 $ util/update_from_stencil 
 [ . . . ]
 Merge made by the 'ort' strategy.
  reference/arm64/vhost   | Bin 2421048 -> 2767416 bytes
  reference/arm64/vrouter | Bin 2421048 -> 2421048 bytes
  reference/vhost         | Bin 2376088 -> 2734744 bytes
  reference/vrouter       | Bin 2400664 -> 2400664 bytes
  4 files changed

If this doesn’t work, check/post on Ed to let us know! Otherwise, if you just want the files, you can go to this repo and download the files in this list, replacing the versions in your repo:

Expand for list of files
  • reference/vhost
  • reference/vrouter
  • reference/arm64/vhost (for Apple silicon macs)
  • reference/arm64/vrouter (for Apple silicon macs)

Once you manually download the files, you will need to make them executable (chmod +x) before you can run them.

  1. Once you’ve updated your files, push the updates to your TCP repo.

Once you’ve pushed your changes, your TCP repo should now be ready and updated to work on TCP, and you should be ready to collaborate, yay! :tada:

Running TCP

In general, you’ll work with TCP the same way as you did with IP, by running networks with vnet_generate and vnet_run. For general resources on how to do this, see the IP getting started guide and the docs and resources.

In TCP, there are a few important differences in how we run networks:

  • We’ll generally only run one network: linear-r1h2! This network has two hosts separated by a single router–since we’re only testing interactions between hosts, we don’t need anything larger!
  • Since the linear-r1h2 network only has one router, your router won’t need to send any RIP messages, so you won’t see any of these in wireshark. Also, if you had issues with RIP, don’t worry, you won’t need it for this project.
  • Usually, you’ll run your vhost with the reference vrouter: since your TCP implementation will be for hosts only, you should use our router to ensure your IP implementation will interoperate with our reference version. A bit later in the project, we’ll release an updated reference vrouter with extra commands for testing lossy networks and debugging–we’ll make an announcement when this is available!

To try out this configuration, and test your wireshark setup, run the linear-r1h2 network and try it with the reference as follows:

  1. Open a terminal in your container and cd to your stencil directory

  2. Generate lnx files for the linear-r1h2 network, like this: util/vnet_generate nets/linear-r1h2 linear-r1h2

  3. Run the network using vnet_run and the reference vhost and vrouter, like this (pick the version based on your host platform):

# Windows, Intel Macs, x86-64 linux
cs1680-user@container:~/tcp-you$ util/vnet_run --router ./reference/vrouter --host ./reference/vhost linear-r1h2

# Apple silicon (M1/M2/M3) macs
cs1680-user@container:~/tcp-you$ util/vnet_run --router ./reference/arm64/vrouter --host ./reference/arm64/vhost linear-r1h2

This should create a tmux session with three panes for r1, h1, and h2. For details on what this would look like and how to work with tmux, see here.

I have an ARM64 Mac and I used the non-arm64 version for IP, can I keep doing that?

You should always run the arm64 version. It turns out that Docker on Apple Silicon macs can automatically run Intel (x86-64) compiled programs using emulation without any special intervention, but this has a significant performance impact (at least 2x slower!). In this project, you’ll measure your implementation’s performance vs. the reference, so you’ll need to run run the reference version without emulation to make sure you have an accurate comparison.

Note: This is a slightly different way to use vnet_run than we did in IP. Going forward, we recommend using --host and --router to run your implementation, as it’s the fastest way to run your vhost with our vrouter. For example, when you go to run your version, you can run it like this:

$ util/vnet_run --host ./vhost --router reference/vrouter linear-r1h2

All of the old ways to use vnet_run will still work, we just recommend remembering this one. For more info on how to use it, see here.

Testing out TCP in Wireshark

Warning: The wireshark setup you need for TCP is slightly different than what you used for IP, so do not skip steps here!

With the reference version open (from the previous section), we can test out TCP! To do this:

  1. If you have not done so already, start the linear-r1h2 network as in the previous section.

  2. In another container terminal, start up wireshark and start a capture on the loopback interface (lo) and enter the capture filter udp port 5000. For details on how to do this, see here (just substitute the new capture filter), as shown in the figure:

In the terminal for h1 or h2 in tmux: run help. This should show you a much larger list of commands, like the example below. These are the new TCP REPL commands!

> help
	[ . . .]
      send Send test packet
        ls List sockets
         a Listen on a port and accept new connections
         c Connect to a TCP socket
         s Send on a socket
         r Receive on a socket
        sf Send a file
        rf Receive a file
        cl Close socket

To start working with sockets, enter some commands! Here’s a recommend starting workflow.

  1. On h2, enter a 9999. This opens up a listen socket on port 9999 and waits for new connections. (Run ls on h2 to see it in the sockets table.)

  2. On h1, connect to h2’s socket by running: c 10.1.0.2 9999. You should see a message printed in h2’s terminal. Run ls on both h1 and h2 to see the new socket that was created, which should look like this:

At the same time, you should see a TCP handshake in wireshark, matching like the figure below (and lecture 12)!

If your wireshark does not look like this
  • If you see only UDP packets (eg. 5000->5001), make sure you have added Decode As… rules for ports 5000-5002.
  • If you see lots of extra packets and/or the error TCP Out-of-Order, make sure your capture filter is set correctly, per step 2 here.

A TCP handshake like this should roughly be your goal for Milestone 1 (in addition to being able to handle multiple sockets).

  1. Now, it’s time to send some packets! For testing, we do this with the s and r commands. Try out the example in the figure below, which does two sends from h1, and then receives them on h2. The result should look like this:

Notice how h2 reassembled the two separate messages that h1 sent, in order! This should roughly be your goal for Milestone 2!

Next steps

Now that you’re able to run TCP, start thinking about your design for handling sockets and your socket API. For the milestone, we’re interested in what data structures you’ll create to represent your sockets and your TCP stack, and how the API functions will create them.

For links to more info, make sure you see the Essential resources above. We’ll also post more resources here about how to work with the reference and tools soon!

Finally, remember that you can still use all the tools and resources and wireshark guide from IP!

Working with the reference

As you think about your implementation, continue trying things out with the TCP reference (while watching in wireshark) to get a sense of how your implementation should behave.

Please note that we switched to a new new TCP reference last year–this is a big deal, since our previous reference was 8+ years old. As we continue testing our reference, we will keep a list of known bugs here. Please take a look at this list if you notice any bugs or behavior that contradicts the specification. For other issues you notice, please post on Ed!