Category Archives: Linux

/proc and process

Leave a reply

Everything is awesome! I mean, in Linux, everything is a file! We even saw where devices are files. But we also have processes, things that run and do things. Are they files? Well, the answer is… (drum roll)…yes. Everything is a file. Where devices are in /dev, process are in /proc. That kind of makes sense, right?

If you ls /, the top directory on your system, you’ll see something like:

bin   dev  home  lost+found  misc  net  opt   root  selinux  sys  usr
boot  etc  lib   media       mnt   od   proc  sbin  srv      tmp  var

So let’s cd into /proc and look around:

1     13     19     2705  2957   328  6          devices      kmsg          softirqs
10    14     2      2769  2972   329  6960       diskstats    kpagecount    stat
1013  14761  20     2771  2985   33   6962       dma          kpageflags    swaps
1043  14766  21     2791  2986   330  6963       drbd         loadavg       sys
1067  15     22     2794  2988   331  7          driver       locks         sysrq-trigger
1076  158    23     28    3      332  8          execdomains  mdstat        sysvipc
1095  16     24     2803  314    333  9          filesystems  meminfo       timer_list
11    160    24853  2841  315    334  901        fs           misc          tty
1102  162    25     2882  316    335  buddyinfo  interrupts   modules       uptime
1150  17     25077  29    320    336  bus        iomem        mounts        version
1152  171    25265  2901  32082  337  cmdline    ioports      net           vmallocinfo
1153  1743   255    2915  322    338  config.gz  irq          pagetypeinfo  vmstat
1161  1745   257    2932  325    339  consoles   kallsyms     partitions    zoneinfo
1173  1746   26     2940  326    34   cpuinfo    kcore        self
12    18     27     2949  327    5    crypto     key-users    slabinf

 

The numbers in proc correspond to processes that are currently running on your Linux system. On this system, there’s a process numbered 29 and one numbered 2901. There’s even 1150, 1152 and 1153. This system isn’t that busy, there’s only 98 processes currently running on it. That may seem like a lot, but a busy web server or a busy mail server could have a thousand busy processes.

Each one of those numbers is a directory and in that directory are files (see what I did there) that are information for that process. I looked in /proc/1 and saw:

ls: cannot read symbolic link 1/cwd: Permission denied
ls: cannot read symbolic link 1/root: Permission denied
ls: cannot read symbolic link 1/exe: Permission denied
attr/       coredump_filter  fdinfo/   mem         oom_adj        root@      status
auxv        cwd@             io        mountinfo   oom_score      sessionid  syscall
clear_refs  environ          limits    mounts      oom_score_adj  smaps      task/
cmdline     exe@             loginuid  mountstats  pagemap        stat       wchan
comm        fd/              maps      net/        personality    statm

What do those first three lines mean? Well, they mean I don’t really have permission to loo at everything in this directory. I could see that further if I did an <b>ls -l</b> on it, but this is really just to show you what’s in the directory.

Next time we’ll talk about some of the files in the /proc directory.

There’s another way

Leave a reply

Back in this post I said ‘there’s more than one way to do it’ should be a motto of Linux©.  We’re going to do it again, this time with memory.  There’s another way to do it.

There’s the free command that shows you how much you have:

Alpha:~ computerlamp$ free
             total       used       free     shared    buffers     cached
Mem:        509688     491316      18372          0      97264     208964
-/+ buffers/cache:     185088     324600
Swap:       262140        316     261824

And we can give it a flag to show that information in gigabytes:

Alpha:~ computerlamp$ free -g
             total       used       free     shared    buffers     cached
Mem:             0          0          0          0          0          0
-/+ buffers/cache:          0          0
Swap:            0          0          0

Wait, something’s wrong.  There’s no memory on this system?  Well, it turns out that this one is an older Alpha.  (I logged into the wrong system this morning) and it doesn’t even have a gigabyte of memory.  I have to give it the -m flag.

Alpha:~ computerlamp$ free -m
             total       used       free     shared    buffers     cached
Mem:           497        479         18          0         94        204
-/+ buffers/cache:        180        317
Swap:          255          0        255

I really should have logged into the new Alpha and not the old one.

Anyway, back to the old Alpha.  There’s another way to look at memory, this time we use the command vmstat.

Alpha:~ computerlamp$ vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0    316  18488  97276 209068    0    0     0     0    1    1  0  0 100  0  0

The columns under the memory tag show us how much we have sitting around.

So, there’s another way.  Always another way in Linux©.

Fun Linux® Commands

Leave a reply

Let’s have some fun with Linux® commands!  There’s lots of interesting and weird commands on most Linux® systems, so lets’ have some fun and try out some fun Linux® commands.

We start with one called figlet.  I know what you’re thinking, who names these things?  At least with Dr. Strange you know exactly what’ you’re getting, you’re getting a strange doctor.  figlet just sounds weird.

 

Dr Strange Clipart

Image from clipartfest.com

 

Let’s see what we get when we try it:

Alpha:~ computerlamp$ figlet test
  m                    m
 mm#mm   mmm    mmm   mm#mm
   #    #"  #  #   "    #
   #    #""""   """m    #
   "mm  "#mm"  "mmm"    "mm

Well isn’t that neat?  It took our word and turned into a banner made of characters.  Now I’m not saying this isn’t a useful command at some point, because clearly someone made it.  I’m saying it is a rather silly command.  It’s also one of many fun Linux® commands!

There isn’t a man page for figlet, but if you look at the output of figlet -h you’ll see all sorts of flags you can use with it.

Let’s do one more.

Alpha:~ computerlamp$ figlet Hi There

 m    m   "          mmmmmmm #
 #    # mmm             #    # mm    mmm    m mm   mmm
 #mmmm#   #             #    #"  #  #"  #   #"  " #"  #
 #    #   #             #    #   #  #""""   #     #""""
 #    # mm#mm           #    #   #  "#mm"   #     "#mm"

That was fun!

There’s another way to do it, of course.  That’s called banner and it’s another of the fun Linux® commands.  It makes similar messages, but they look a little different:

Alpha:~ computerlamp$ banner -w 40
Message: Hi There
          #                         #
          #                         #
          ###########################
          ###########################
          #              #          #
          #              #          #
                         #
                         #
                         #
          #              #          #
          ###########################
          ###########################
          ###########################
          #                         #

          #              #
          ################    ####
          ################    ####
          ################     ##











                                #####
                                #####
                                   ##
                                    #
          #                         #
          #                         #
          ###########################
          ###########################
          #                         #
          #                         #
                                    #
                                   ##
                                #####
                                #####
          #                         #
          ###########################
          ###########################
          ###########################
                        #
                        ##
                        ##
          ################
          ###############
          ############
               #####
            ###########
           ##############
          ##     #     ##
          #      #       #
          #      #       #
          #      #      ##
           #     ########
            ##   ######
                 ###
          #              #
          ################
          ################
          ################
                      ##
                       ##
                     #####
                    ######
                    #####

               #####
            ###########
           ##############
          ##     #     ##
          #      #       #
          #      #       #
          #      #      ##
           #     ########
            ##   ######
                 ###

When I ran the banner command, I gave it that -w flag so that it wouldn’t fill the entire string.  Try it without any flag and see what you get.  If you don’t pass in the string for the message, it asks you what you want to use.

IP Address Origins

Leave a reply

You know how to find your IP address, right? You can just run ifconfig and it will tell you.  Now the big question: How did your computer get that IP address? It didn’t buy one at the IP store or pull one out of the closet, it had to get it somehow. Well, there’s two places it
could have come from, and in this post we’ll talk about where your IP address came from.

The first choice is called a static assignment. That means your ISP picked an IP from the collection it has and assigned it to you.

Home address

Home address

It’s like the address of your house. It isn’t going to change, it’s always going to be that address. Unless you move your house completely, then your house address will change. If you move ISPs, then your IP address would change.

This used to be the way IP addresses were assigned to everyone. Remember how I said we’re running out of IP addresses? That doesn’t work as well any more. Now we have a dynamic pool.

Dead Pool

Dead Pool

Not Dead Pool, but a dynamic pool of IP addresses. When you tell your ISP ‘Hey, I’m going online’ it takes an IP address from its pool and assigns it to you. When you turn off your computer or your router, you give that IP address back to the pool.

Swimming Pool

Swimming Pool

Well, not into the swimming pool but rather the collection of IP addresses.

This is called DHCP, or Dynamic Host Configuration Protocol. That’s quite the mouthful, so we’ll stick with DHCP.

DHCP usually comes with a timer. That means you have to renew your IP address periodically. Normally it just means you say ‘yes, I’m here, I’m still using it, go away’ but depending on how the configuration works, you might get a new IP address. In other words, your IP addresses aren’t set in stone like your house address, but are likely to change. Like if you had a houseboat and moved it often, then that house address would change.

Devices

Leave a reply

Way back at the beginning I said that Linux® treats everything as a file.  Everything!  That includes devices, like the network interface, the wireless interface, the USB port, the motherboard, the sound board, speakers, microphones, you get the idea.  Every device you could add to your computer is treated like a file.

We can’t have those files floating around willy nilly, can we?  That’d be very confusing very quickly.  Sort of like asking the Hulk to keep track of Hawkeye’s arrows.  it would go really badly.

Hawkeye and His Arrows

Hawkeye

Instead, Linux® keeps all those files in one place, known as the /dev directory.  Every device has a file in this directory that Linux® uses to communicate with it.

If I do ls -l on /dev I get 676 lines.  I know because I did this:

Alpha:~ computerlamp$ ls -l /dev |  wc -l 
676

That doesn’t mean there are 676 devices on my system!  I’d have one busy system if that was the case.   It means that there are the possibilities for that many devices.

Let’s look at some of the output from ls -l:

Alpha:~ computerlamp$ ls -l /dev/
crw------- 1 root root  10, 235 Aug 11 02:19 autofs
crw------- 1 root root   5,   1 Aug 11 02:19 console
lrwxrwxrwx 1 root root       11 Aug 11 02:19 core -> /proc/kcore
crw------- 1 root root  10,  58 Aug 11 02:19 cpu_dma_latency
drwxr-xr-x 3 root root       60 Aug 11 02:19 disk
crw------- 1 root root  10,  63 Aug 11 02:19 dlm-control
crw------- 1 root root  10,  62 Aug 11 02:19 dlm-monitor
crw------- 1 root root  10,  61 Aug 11 02:19 dlm_plock
crw------- 1 root root  10,  60 Aug 11 02:19 ecryptfs
lrwxrwxrwx 1 root root       13 Aug 11 02:19 fd -> /proc/self/fd
crw-rw-rw- 1 root root   1,   7 Aug 11 02:19 full
crw------- 1 root root  10, 229 Aug 11 02:19 fuse

That isn’t very helpful, is it.  I don’t know what dlm means and why there’s a dlm-control or a dlm-monitor  I’m going to guess that the disk directory has something to do with hard drives, but I’m not sure.  encryptfs looks interesting, I bet it has something to do with encrypting things, but I’m not sure of that either.

The other thing to pay attention to is that the ownership and permissions of a lot of the files in this directory are restricted to root.  That means that while I can use ls -l to see that they’re there, I can’t actually look at them.

More than one way to do it

Leave a reply

There’s more than one way to do it!  I think that should be one of the mottos of Linux®.  There’s always more than one way to do it.  That’s what we’re going to talk about in this post, some of the more than one way to do the commands we’ve already learned.

We learned about ls and how it has flags.  One of the most useful flags is -l which lets us see all the group ownerships and users and everything we could need.

For example:

Alpha:~ computerlamp$ ls -l
total 164
drwxr-xr-x   3 root root 69632 Aug  9 13:15 bin
drwxr-xr-x   2 root root  4096 Jan  5  2016 games
drwxr-xr-x  82 root root 20480 Jul 17 16:18 include
drwxr-xr-x 111 root root 24576 Jul 17 16:18 lib
drwxr-xr-x   3 root root 12288 Aug  3 06:43 libexec
drwxr-xr-x  19 root root  4096 Dec 27  2016 local
drwxr-xr-x   2 root root 12288 Jul 28 18:58 sbin
drwxr-xr-x 199 root root  4096 Jul 28 18:58 share
drwxr-xr-x   2 root root  4096 Apr 19  2012 src
drwxr-x---   3 root root  4096 Jan  5  2016 var

I listed all the files and directories in /usr using ls -l.  I can do the same thing with the command vdir.

Alpha:~ computerlamp$ vdir /usr
total 164
drwxr-xr-x   3 root root 69632 Aug  9 13:15 bin
drwxr-xr-x   2 root root  4096 Jan  5  2016 games
drwxr-xr-x  82 root root 20480 Jul 17 16:18 include
drwxr-xr-x 111 root root 24576 Jul 17 16:18 lib
drwxr-xr-x   3 root root 12288 Aug  3 06:43 libexec
drwxr-xr-x  19 root root  4096 Dec 27  2016 local
drwxr-xr-x   2 root root 12288 Jul 28 18:58 sbin
drwxr-xr-x 199 root root  4096 Jul 28 18:58 share
drwxr-xr-x   2 root root  4096 Apr 19  2012 src
drwxr-x---   3 root root  4096 Jan  5  2016 var

There’s more than one way to list the files.  I can even do it with find though it’s much more work than just doing vdir or ls.

If I leave off the -l flag, I get:

Alpha:~ computerlamp$ ls /usr
bin  games  include  lib  libexec  local  sbin	share  src  var

I get the same output if I do the following two commands, just with fewer spaces:

Alpha:~ computerlamp$ cd /usr
Alpha:~ computerlamp$ echo *

In other words, there’s more than one way to do it.  I can ls -l or vdir, or I can ls or use the echo * trick.

This is certainly not the only case of more than one command to do the same thing.  It keeps things interesting when you’re learning Linux®.

IPv6 addresses

Leave a reply

IP addresses are formally called IPv4 addresses.  This post is to disuss the next generation of IP addresses, called IPv6 addresses.

This is kind of like Star Trek: The Next Generation, but without Data and Picard and the whole crew.

 Ok, it’s nothing like them.

In this post I said that there are 232 IP addresses, or 4,294,967,296 IP addresses.  Some of these are reserved and shouldn’t be used on the Internet, but that’s still a lot of IP addresses, right?  Enough to almost give every person in the world their very own IP address.  That leaves out companies having a whole bunch of computers on the Internet, or Universities with their computers, or even light bulbs with their IP addresses.  We’re running out of IP addresses!

The smart people that work on engineering the Internet, known as the IETF decided to solve this problem.  On a side note, they really could have come up with a better name.  Maybe Super Hardworking Internet Engineers Leading Development.  That sounds awesome, right?  They’d have to come up with their own Nick Fury though.

Nick Fury

Nick Fury

Back to the problem.   232 IP addresses aren’t enough.  The geniuses at the IETF decided not to just double that number, but to square it, and then square that number.  That gives us 2128 IP addresses, or 340,282,366,920,938,463,463,374,607,431,768,211,456 IP addresses.

Have fun reading that number out loud!

They also changed the notation for the IP addresses.  They don’t look like IPv4 addresses, they use : instead of . to separate numbers.  They also use base 16 instead of base 10 for the addresses.  Bet you didn’t think there would be math in this blog, did ya.

Here’s an example of an IPv6 address:

2601:547:902:cba7:4b8:ff43:f419:cd01

That’s a mouthful to remember.  I have an easier time remembering 192.168.0.1.

The IPv6 protocol that the SHIELD, okay, IETF engineers designed does a lot more than just creating the IP addresses, but that’s all we’re concerned with for right now.

Now remember that ifconfig output?

eth0      Link encap:Ethernet  HWaddr 00:1A:2B:3C:00:00
          inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fd91:ae3b:d5e8:aa85:0:0:0:0 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:41620 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40231 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21601203 (20.6 MiB)  TX bytes:6145876 (5.8 MiB)
          Interrupt:21 Base address:0xe000

The third line is the IPv6 address of the system.  A system can have both kinds on one interface.  Neat, huh.

More ifconfig!

Leave a reply

We’ve talked about IPv6, we’ve talked about broadcast and other parts of the ifconfig output, but there’s more ifconfig output to look at.   The rest of the story, as it were.  So we have this output:

eth0      Link encap:Ethernet  HWaddr 00:1A:2B:3C:00:00
          inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fd91:ae3b:d5e8:aa85:0:0:0:0 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:41620 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40231 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21601203 (20.6 MiB)  TX bytes:6145876 (5.8 MiB)
          Interrupt:21 Base address:0xe000

eth0 is the interface for your computer to the internet.  That thing that all the traffic goes through.  That means that there’s Internet traffic going through that device.   ifconfig shows us more information than just the IP address.

Network Card 2 clip art

Let’s skip to these two lines:

          RX packets:41620 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40231 errors:0 dropped:0 overruns:0 carrier:0

The Internet breaks all traffic up into packages called packets.  Think of it like a really really long text.   Every 160 characters in your text, your phone will send along the message and wait for the next bit.  (That 160 characters depends on your phone.  It could be more, it could be less.)  Your computer does the same thing.  It takes your messages and breaks them into chunks and sends it through the Interface.  It can also receive things through the interface.

The RX packets packages string in the ifconfig output is the number of packets your computer received and the TX packets is the number of packets your computer transmits.

Now these lines:

RX bytes:21601203 (20.6 MiB)  TX bytes:6145876 (5.8 MiB)

Are the total amount of traffic that went through all those pipes.  We received 20.6 megabytes of data and we sent out 5.8 megabytes of data.  We were busy surfing the web and pulled down the fun stuff.

The last bit of the output that we’re going to talk about is this line:

collisions:0

A collision occurs when two devices on the same network try to transmit at the same time.  ifconfig keeps track of this because it usually means that there’s something wrong with the network and it should be looked at.

Now we’ve discussed all the important parts of the network interface!  If you want more information, read this site.

 

ICMP

Leave a reply

If the Avengers: Age of Ultron taught us anything, its that sometimes ideas fail spectacularly. Case in point, Ultron wanted to destroy the world to save that. That was a big mistake by Tony Stark, and not just a big mistake, a  BIG BIG mistake. On the Internet, sometimes things fail as well, just not as spectacularly as Ultron, but still, a failure. There’s a protocol to tell us that something went wrong, it’s called ICMP. It’s not nearly as dangerous as Ultron trying to make the world explode, and that’s a good thing.

Ultron

Ultron

In this post I said there were 65535 ports on a system. What happens if you try to access port 99999? That port doesn’t exist!

ERROR ERROR ERROR

ERROR ERROR

 

The good news is that your computer won’t blow up. That would be a good plot for a movie, don’t you think? Send a connection to a system on a particular port and watch the computer go BOOM. (Hollywood, if you use that, I thought of it first.)

The other good news is that you won’t even notice it when it happen, unless you’re sitting there watching every little bit of traffic coming to your computer (if so you need a new hobby). Your computer silently handles the problem by sending an ICMP packet saying ‘Nope, that’s not here.’

Your computer can also send ICMP packets out. The program ping uses what’s called an ICMP ECHO_REQUEST packet to see if another computer is alive and to find out how long it takes to get there. It runs like:

 

Alpha:~ computerlamp$ ping www.google.com
PING www.google.com (173.194.205.103): 56 data bytes
64 bytes from 173.194.205.103: icmp_seq=0 ttl=45 time=27.505 ms
64 bytes from 173.194.205.103: icmp_seq=1 ttl=45 time=24.952 ms
64 bytes from 173.194.205.103: icmp_seq=2 ttl=45 time=23.545 ms
64 bytes from 173.194.205.103: icmp_seq=3 ttl=45 time=24.521 ms
64 bytes from 173.194.205.103: icmp_seq=4 ttl=45 time=23.499 ms
64 bytes from 173.194.205.103: icmp_seq=5 ttl=45 time=27.609 ms

It keeps going and going and going… until you use Control-C to stop it.

When I do hit control-C, it tells me some statistics about the traffic:

--- www.google.com ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 23.499/25.272/27.609/1.695 ms

Or, I can use the -c flag for ping. That way it will only do the number of pings I tell it to:

Alpha:~ computerlamp$ ping www.google.com
PING www.google.com (173.194.205.103): 56 data bytes
64 bytes from 173.194.205.103: icmp_seq=0 ttl=45 time=24.166 ms
64 bytes from 173.194.205.103: icmp_seq=1 ttl=45 time=23.373 ms
64 bytes from 173.194.205.103: icmp_seq=2 ttl=45 time=23.462 ms
64 bytes from 173.194.205.103: icmp_seq=3 ttl=45 time=23.437 ms
64 bytes from 173.194.205.103: icmp_seq=4 ttl=45 time=23.783 ms

--- www.google.com ping statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 23.373/23.644/24.166/0.297 ms

 

How TCP Really Works

Leave a reply

I talked about TCP and UDP in this post. Now we’re going to talk about how TCP really works, how it keeps that connection going. It’s how mail works, it’s how a lot of web traffic works, so how does that pipe keep going?

 

A pipe

A pipe

Well, it isn’t a physical pipe like that picture. It isn’t a physical connection at all and it also isn’t a psychic connection, instead, it’s a virtual connection.

It starts by the two systems, let’s call them Captain America and Black Panther, negotiating a connection.  The connection is called a handshake.

A handshake

A handshake

Actually it’s called a three-way handshake and it works like this:

Captain America tells Black Panther, “Hey, I want to talk to you.”. He does this by sending a packet called a SYN packet.

Black Panther then tells Captain America, “Gotcha! Good idea, Let’s talk.” He does this by sending back a packet called an ACK. The three-way is because Captain America acknowledges Black Panther’s agreement to talk by sending back his own ACK.

The TCP connection uses these three packets to set the connection up, SYN, ACK and ACK.

So they’re talking away, but like in all conversations, sometimes there’s a lull where neither side says anything. Maybe they’re each getting a nice drink, maybe they’re each writing a note to themselves to make fun of Iron Man, maybe they’re just taking a nap. The connection sends back and forth KEEP ALIVE packets to say ‘Hey, you still there?’ and the other side acknowledges ‘Yup, still here. Taking a nap, be back to talking with you in a minute.’

All conversations eventually end, or at least take enough of a break that you hang up the phone and do something else for a while. When this happens, the TCP connection is shutdown, like when you hang up your phone. Your phone tells the other guy ‘yup, I’m done’ and shuts down the communication. Similarly, the one end of the TCP connection will tell the other ‘I’m outta here’, which ends the connection.