More ifconfig!

We’ve talked about IPv6, we’ve talked about broadcast and other parts of the ifconfig output, but there’s more ifconfig output to look at.   The rest of the story, as it were.  So we have this output:

eth0      Link encap:Ethernet  HWaddr 00:1A:2B:3C:00:00
          inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fd91:ae3b:d5e8:aa85:0:0:0:0 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:41620 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40231 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21601203 (20.6 MiB)  TX bytes:6145876 (5.8 MiB)
          Interrupt:21 Base address:0xe000

eth0 is the interface for your computer to the internet.  That thing that all the traffic goes through.  That means that there’s Internet traffic going through that device.   ifconfig shows us more information than just the IP address.

Network Card 2 clip art

Let’s skip to these two lines:

          RX packets:41620 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40231 errors:0 dropped:0 overruns:0 carrier:0

The Internet breaks all traffic up into packages called packets.  Think of it like a really really long text.   Every 160 characters in your text, your phone will send along the message and wait for the next bit.  (That 160 characters depends on your phone.  It could be more, it could be less.)  Your computer does the same thing.  It takes your messages and breaks them into chunks and sends it through the Interface.  It can also receive things through the interface.

The RX packets packages string in the ifconfig output is the number of packets your computer received and the TX packets is the number of packets your computer transmits.

Now these lines:

RX bytes:21601203 (20.6 MiB)  TX bytes:6145876 (5.8 MiB)

Are the total amount of traffic that went through all those pipes.  We received 20.6 megabytes of data and we sent out 5.8 megabytes of data.  We were busy surfing the web and pulled down the fun stuff.

The last bit of the output that we’re going to talk about is this line:

collisions:0 

A collision occurs when two devices on the same network try to transmit at the same time.  ifconfig keeps track of this because it usually means that there’s something wrong with the network and it should be looked at.

Now we’ve discussed all the important parts of the network interface!  If you want more information, read this site.

 

Interface

I talked about ports and services and now I’ll talk about the interface.  The interface is the actual network connection that has all that traffic to the ports come through it.  It’s that ethernet cable attached to your computer.

Ethernet Cable

Ethernet Cable

Or it’s that connection to the wireless network.  It’s the pipe (not a real pipe!) that connects your computer to the Internet.  Your computer has one, since you’re reading this blog post on the Internet.  Since we’re using Linux® there’s a way to see what interfaces you have.  It’s called ifconfig.

If you try to run ifconfig at the commandline, you’ll see:

Alpha:~ computerlamp$ ifconfig
-bash: ifconfig: command not found

That means it isn’t in your path.  (We’ll talk more about that later).  It just means that you have to give the full command line to see the output:

 

Alpha:~ computerlamp$ /sbin/ifconfig
lo        Link encap:Local Loopback
         inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:480 (480.0 b)  TX bytes:480 (480.0 b)

eth0      Link encap:Ethernet  HWaddr 00:1A:2B:3C:00:00
          inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fd91:ae3b:d5e8:aa85:0:0:0:0 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:41620 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40231 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21601203 (20.6 MiB)  TX bytes:6145876 (5.8 MiB)
          Interrupt:21 Base address:0xe000

You have at least two interfaces on your system, the first one is your local link.  That’s the interface that your computer talks to itself on.  It isn’t a real interface, you don’t plug a cable into it, but it’s the basic ‘I’m talking to myself!’  interface that every computer has.  The second interface is eth0, or ethernet 0.  If you have two ethernet interfaces (meaning you plug two ethernet cables into your computer) you’ll see eth0 and eth1.  And then things can get weirder.

Wireless interfaces don’t have a standard.  They can be named all sorts of things.

Next time, we’ll talk about what the actual output from ifconfig means.

ICMP

If the Avengers: Age of Ultron taught us anything, its that sometimes ideas fail spectacularly. Case in point, Ultron wanted to destroy the world to save that. That was a big mistake by Tony Stark, and not just a big mistake, a  BIG BIG mistake. On the Internet, sometimes things fail as well, just not as spectacularly as Ultron, but still, a failure. There’s a protocol to tell us that something went wrong, it’s called ICMP. It’s not nearly as dangerous as Ultron trying to make the world explode, and that’s a good thing.

Ultron

Ultron

In this post I said there were 65535 ports on a system. What happens if you try to access port 99999? That port doesn’t exist!

ERROR ERROR ERROR

ERROR ERROR

 

The good news is that your computer won’t blow up. That would be a good plot for a movie, don’t you think? Send a connection to a system on a particular port and watch the computer go BOOM. (Hollywood, if you use that, I thought of it first.)

The other good news is that you won’t even notice it when it happen, unless you’re sitting there watching every little bit of traffic coming to your computer (if so you need a new hobby). Your computer silently handles the problem by sending an ICMP packet saying ‘Nope, that’s not here.’

Your computer can also send ICMP packets out. The program ping uses what’s called an ICMP ECHO_REQUEST packet to see if another computer is alive and to find out how long it takes to get there. It runs like:

 

Alpha:~ computerlamp$ ping www.google.com
PING www.google.com (173.194.205.103): 56 data bytes
64 bytes from 173.194.205.103: icmp_seq=0 ttl=45 time=27.505 ms
64 bytes from 173.194.205.103: icmp_seq=1 ttl=45 time=24.952 ms
64 bytes from 173.194.205.103: icmp_seq=2 ttl=45 time=23.545 ms
64 bytes from 173.194.205.103: icmp_seq=3 ttl=45 time=24.521 ms
64 bytes from 173.194.205.103: icmp_seq=4 ttl=45 time=23.499 ms
64 bytes from 173.194.205.103: icmp_seq=5 ttl=45 time=27.609 ms

It keeps going and going and going… until you use Control-C to stop it.

When I do hit control-C, it tells me some statistics about the traffic:

--- www.google.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 23.499/25.272/27.609/1.695 ms

Or, I can use the -c flag for ping. That way it will only do the number of pings I tell it to:

Alpha:~ computerlamp$ ping www.google.com
PING www.google.com (173.194.205.103): 56 data bytes
64 bytes from 173.194.205.103: icmp_seq=0 ttl=45 time=24.166 ms
64 bytes from 173.194.205.103: icmp_seq=1 ttl=45 time=23.373 ms
64 bytes from 173.194.205.103: icmp_seq=2 ttl=45 time=23.462 ms
64 bytes from 173.194.205.103: icmp_seq=3 ttl=45 time=23.437 ms
64 bytes from 173.194.205.103: icmp_seq=4 ttl=45 time=23.783 ms

--- www.google.com ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 23.373/23.644/24.166/0.297 ms

 

Services

Remember the post where I mentioned /etc/services? Let’s talk more about that file.  It’s a list of all possible services you could run on your Linux® system, it doesn’t mean that your system is running those.

What port do web services use?  Well, web services are formally known as http, or HyperText Transport Protocol.  We’ll just call it http.   We can grep for http in /etc/services by:

Alpha:~ computerlamp$ grep http /etc/services

I got 105 lines back from that simple command.  I know that because I did this command when the first one scrolled a lot:

Alpha:~ computerlamp$ grep http /etc/services | wc -l

That’s a lot of ports.  Let’s pipe that command thorugh less and see what we get:

Alpha:~ computerlamp$ grep http /etc/services | less
#       http://www.iana.org/assignments/port-numbers
http             80/udp     www www-http # World Wide Web HTTP
http             80/tcp     www www-http # World Wide Web HTTP
http-mgmt       280/udp     # http-mgmt
http-mgmt       280/tcp     # http-mgmt
https           443/udp     # http protocol over TLS/SSL
https           443/tcp     # http protocol over TLS/SSL
gss-http        488/udp     # gss-http
gss-http        488/tcp     # gss-http
http-alt        591/udp     # FileMaker, Inc. - HTTP Alternate (see Port 80)
http-alt        591/tcp     # FileMaker, Inc. - HTTP Alternate (see Port 80)
http-rpc-epmap  593/udp     # HTTP RPC Ep Map
http-rpc-epmap  593/tcp     # HTTP RPC Ep Map
multiling-http  777/udp     # Multiling HTTP
multiling-http  777/tcp     # Multiling HTTP
netconfsoaphttp 832/tcp     # NETCONF for SOAP over HTTPS
netconfsoaphttp 832/udp     # NETCONF for SOAP over HTTPS
llsurfup-http   1183/udp    # LL Surfup HTTP
llsurfup-http   1183/tcp    # LL Surfup HTTP
llsurfup-https  1184/udp    # LL Surfup HTTPS
llsurfup-https  1184/tcp    # LL Surfup HTTPS
compaq-https    2381/udp    # Compaq HTTPS

The first line is a comment and that points us to the website that has all the registered ports.

The second line is the UDP port for HTTP, which is port 80.  TCP uses the same port.  That’s the basic port that all web traffic goes over, unless it is encrypted.  If it’s encrypted, it uses port 443.  That’s what this sentence means ‘http protocol over TLS/SSL’, it means ‘http traffic that’s encrypted’.

 

How TCP Really Works

I talked about TCP and UDP in this post. Now we’re going to talk about how TCP really works, how it keeps that connection going. It’s how mail works, it’s how a lot of web traffic works, so how does that pipe keep going?

 

A pipe

A pipe

Well, it isn’t a physical pipe like that picture. It isn’t a physical connection at all and it also isn’t a psychic connection, instead, it’s a virtual connection.

It starts by the two systems, let’s call them Captain America and Black Panther, negotiating a connection.  The connection is called a handshake.

A handshake

A handshake

Actually it’s called a three-way handshake and it works like this:

Captain America tells Black Panther, “Hey, I want to talk to you.”. He does this by sending a packet called a SYN packet.

Black Panther then tells Captain America, “Gotcha! Good idea, Let’s talk.” He does this by sending back a packet called an ACK. The three-way is because Captain America acknowledges Black Panther’s agreement to talk by sending back his own ACK.

The TCP connection uses these three packets to set the connection up, SYN, ACK and ACK.

So they’re talking away, but like in all conversations, sometimes there’s a lull where neither side says anything. Maybe they’re each getting a nice drink, maybe they’re each writing a note to themselves to make fun of Iron Man, maybe they’re just taking a nap. The connection sends back and forth KEEP ALIVE packets to say ‘Hey, you still there?’ and the other side acknowledges ‘Yup, still here. Taking a nap, be back to talking with you in a minute.’

All conversations eventually end, or at least take enough of a break that you hang up the phone and do something else for a while. When this happens, the TCP connection is shutdown, like when you hang up your phone. Your phone tells the other guy ‘yup, I’m done’ and shuts down the communication. Similarly, the one end of the TCP connection will tell the other ‘I’m outta here’, which ends the connection.

TCP and UDP the Backbone of Traffic

In the previous post we talked about the /etc/services file and how that told the computer what ports to use for what traffic.  In that file we saw lots of mentions of tcp and udp but there wasn’t anything that told you what those two abbreviations meant.

In the Internet, there’s different destinations for traffic and there’s also different kinds of traffic.  They’re designed for two different things.

When you send a letter by snail mail, you open up the mailbox, drop your letter in, and hope it gets there.  It usually makes it to its destination, but you have no way of making sure it does.

Mailbox

Mailbox

On the other hand, when you make a phone call, you know it connects (because you’re talking to the other guy) and it works until either you hang up or the connection cuts off for some reason.  You don’t always know why it got interrupted, just that something cut it off.

Telephone

Telephone

We can communicate two ways, each of them designed for some specific reason.

In the Internet, udp traffic is like the mailbox.  Its traffic that just flows across the Internet with no verification that it made it.  It’s used in case when it’s not really a huge deal if the connection doesn’t work.  Network Time Protocol is an example of udp traffic.  It isn’t great if the time update doesn’t make it, but it isn’t the end of the world either.

On the other hand, tcp traffic is like the telephone call.  In tcp traffic, a connection is created between two systems and it is kept alive as long as they need it.  It’s for when you really want your traffic to make it from one system to another.  E-mail  uses tcp, because as I said before, people really want their email.

Network Traffic

You’re on the network, surfing away on the web, and there’s data just flowing to your box.  Even right now, when you’re looking at this web page, there’s data heading to your system. It’s network traffic and if you think of it as a pipe, your system has that flowing in all the time.  Imagine it as a water pipe, like this picture:

Leaky Pipe

Leaky Pipe

We’ll pretend our pipe isn’t leaking, since that would mean we’re losing network traffic.  Our pipe brings us all kinds of traffic as we surf away, some of which we want and some of which we don’t.

How does the computer know what kind of traffic the pipe is bringing it?  Computers are mostly stupid, if you put legs on one and told it to walk off a cliff, it would happily do that. We have to tell the computer what kind of traffic to expect, either DNS or web traffic or email, or other kinds of traffic we’re not mentioning here.

Back to our pipe analogy because it’s a really useful way to think about the traffic. If before the pipe connects to us, it separates itself into different types of traffic, that would work, right?

Many Pipes

Many Pipes

We can attach those little pipes to the pipe in the first picture and that would separate out the traffic.

The way modern operating systems (Linux© included) handle this is with ports.  The pipe connecting you to the network has labeled ports and the different kinds of traffic go to the different ports.  Web traffic goes to port 80, e-mail to port 25, DNS to port 53, and there’s more. Each machine has 65,535 ports on it for traffic to use.

I know you’re asking the next question:  How does the computer know what kinds of port do what kind of traffic?  Well, there’s a file for that.   It’s called /etc/services.  If you more it, you can find see people reserve ports for different kinds of applications.

There’s also a website  that keeps track of assigned ports.

Next time we’ll talk about what tcp and udp in the /etc/services file mean.

 

MUA and MTA

In the last two posts, we’ve talked about how email is handled in DNS. It’s an important first step, you can’t send email if you don’t know where it’s going to go. Now we’re going to talk about two parts of the email system called the MUA and MTA.

The MUA is the Mail User Agent. How’s that for a boring term? It means that program you (the user) are using to send and read email. That page in your web browser, that application on your phone, whatever you’re using, that’s called a MUA. I’ll list a few examples:

  • elm (REALLY OLD)
  • pine (Not as old as elm, but OLD)
  • outlook (Microsoft’s flagship!)
  • your web browser of choice
  • thunderbird (OLD)
  • mail (aka that app on your Mac)

That’s just a few of the programs that can send and read email. Your system doesn’t actually receive it though, that’s the job of the MTA. Your MUA doesn’t do the real job of sending it either, it hands that off to the MTA as well.

So what’s an MTA? An MTA is a Mail Transfer Agent. It’s the computer program that actually sends and receives your email. You don’t see it work, it’s like when you send your email you drop it into a box:

A box

A box

the MTA picks it up and sends it along, like a postman:

Postal Van

Postal Van

And it lands in the other person’s Inbox:

Inbox

Inbox

Of course, that’s not what your inbox looks like, but it’s  a pretty picture.  Your inbox is actually a file whose format depends on the MTA and most MTAs have their own format.

There’s many different MTAs available. Your system uses whatever the system administrator set up, so you don’t have to worry about it. A few of them are:

  • sendmail (one of the originals!)
  • exim (not as old as sendmail)
  • qmail (older than exim but not as old as sendmail)

You don’t have to worry about the set up and management of these systems, someone else does that. If you want to learn, there’s books (and books and books) on the subject.

MX Preference

In the previous post, we talked about MX records. When we looked at gmail.com’s MX records, there were numbers before the hostnames. Those numbers are preferences, also known as MX preference.

How’s that for a technically boring term? MX preference means something when the computer is determining which of the records to use as a destination for my email. Let’s look at those results for gmail.com again:

gmail.com mail is handled by 40 alt4.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 20 alt2.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 30 alt3.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 10 alt1.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 5 gmail-smtp-in.l.google.com.

 

The one with the lowest number has the highest preference. How’s that for a contradictory sentence? But it’s how it works. This means that if I’m sending email to gmail.com, then the first place it’s going to try is the system gmail-smtp-in.l.google.com. If this fails, then it goes to the next in the list by preference, that is, alt2.gmail-smtp-in.l.google.com.

The geniuses (Not Dr. Strange and Ironman!) who designed this knew that people really wanted to get their email. Well, not spam email, but they didn’t consider that. They knew that people really wanted email and it should work, so they made it so what should happen is that the system sending the email should try each in order of MX preference.

Not everyone does it, of course. The Internet is a cooperative and it depends on people doing the ‘right thing’ or an approximation close enough to keep things running. Sometimes you’ll send an email and see a bounce from the mailer-daemon (not a real demon), but if you try again in just a few minutes, it’ll go through.

Either the other domain didn’t have a server up to receive the email or the system didn’t check through the MX preference list like it should have, or there was a real demon in the system. Any one of these could have caused the problem.  Especially the demon, demons are trouble.

Fred the Demon

Fred the Demon

I call mine Fred.

Disclaimer: There are no real demons in the Internet. …I think.

Email and DNS

I get lots of email, some of it I want, some of it I’d rather not see. I’m sure that’s true for everyone with an email address. If I get the email, how does it get there? If I send email to my brother, how does my computer know how to get the email to him? It’s a bit of a complicated process, so we’re going to break it down and start by talking about Email and DNS.

One day, I decide I want to send an email to myfriend@example.com.

Why did I say example.com rather than a domain like gmail.com or hotmail.com? That’s because example.com is a domain that was set aside at the beginning of Internet time so people can use it in examples without referring to existing domains. It’s an example, just like its name.  It’s useful to use it because existing domains have been known to go away or change behavior, but example.com is always that, an example.

Ok, back to the subject at hand!

I’m sending email to myfriend@example.com and my first question is how does my email know where to send it?  My second question is, do I send it to example.com or somewhere else?

The geniuses that set up the Internet originally knew that the owner of a domain wasn’t necessarily the same person that ran the domain. The  domain computerlamp.net is what has this blog and while I own that domain, I don’t run things around here. I let DreamHost (Hi guys!) run this for me. That means that they handle my email, I don’t. All I do is read it.

DNS lets us know this by using the MX record. (MX means ‘mail exchanger’. They skipped the E when they made the acronym.)

The MX record for computerlamp.net is:

computerlamp.net mail is handled by 0 mx2.sub4.homie.mail.dreamhost.com.
computerlamp.net mail is handled by 0 mx1.sub4.homie.mail.dreamhost.com.

I did this using the command host.  Using the -t flag, my command was:

Alpha:~ computerlamp$ host -t MX computerlamp.net

If I wanted to know who handles the email for gmail.com, I’d do:

Alpha:~ computerlamp$ host -t MX gmail.com
gmail.com mail is handled by 40 alt4.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 20 alt2.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 30 alt3.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 10 alt1.gmail-smtp-in.l.google.com.
gmail.com mail is handled by 5 gmail-smtp-in.l.google.com.

I got 5 responses back… with numbers in them! Next time, we’ll talk about what those numbers mean.