Using static IP's on Verizon 4G

First a little background info on IP addresses and cellular data service. 3G data connections use publicly accessible valid internet IP addresses. While this is nice if you want remote access to a device it does needlessly use up increasingly valuable IPv4 addresses. When carriers rolled out their next generation 4G service they switched to using private 10.x.x.x networks and NATed the traffic out to the internet somewhere within their network. It is possible to get publicly accessible static IP's from Verizon but they don't make the process very easy.

Requesting static IP's

I am using Verizon 4G service with Cradlepoint routers as a backup internet connection at my remote offices. I wanted to use static IP addresses so I could get access to these offices if the primary internet connection went down. We have a Verizon business sales rep and he was the person that handled our static IP request. Verizon charges a one time $500 dollar fee to add static IP's to your account. First step was to authorize the one time $500 charge. My accounting department handled that and then my Verizon sales rep sent the request somewhere deep into the bowels of the Verizon bureaucracy. A month later we were approved. Next someone from Verizon called to ask what 'sub-group' or 'level' we wanted these static IP's to be attached to. It took a while on the phone to figure out what exactly they were asking. Turns out we have our Verizon devices setup in two different groups. One group is devices with phone and data service and the other group is data only (things like iPads, hotspots or Cradlepoints). So we applied the static IP's to our data only group of devices. From the conversation I had on the phone with this Verizon person my understanding is that we would need to pay $500 bucks for each group of devices.

Assigning static IP's to 4G devices

It would be really nice if there was some sort of website to assign static IP's to devices but sadly there is not. The process for attaching a static IP to a certain device is to e-mail your Verizon business sales rep the device IMEI and/or the phone number assigned to the device. The sales rep then handles assigning the static IP and will e-mail back the static IP address once one has been assigned to the device.

Configuring 4G devices to use static IP's

This is where information got really nebulous. I asked my Verizon sales rep if I needed to do any configuration to my Cradlepoint router for the static IP. He said "nope, it should just work". Well that is definitely not the case. To be able to use static IP's you must change a setting for something called the APN. The APN is used to identify what network the device should attach to. The ability to change the APN of a device varies depending on the carrier. My AT&T iPhone does not present any options to change that setting but this Apple knowledge base document shows the option does exist. On Cradlepoint routers this option is easily accessible because it is a somewhat common thing to modify on those devices. The APN menu location on Cradlepoints depends on the device but it is usually under either modem settings or the Connection Manager.
Now what should the APN be set to for Verizon devices? Well this took a bit of searching. I found a few blog posts that said it should be set to "mw01.vzwstatic". I tried this and the modem kept dropping it's connection with an error saying carrier rejected. So after more searching I found this list of Verizon APN's:

1. ne01.vzwstatic (NorthEast)

2. nw01.vzwstatic (NorthWest)

3. so01.vzwstatic (South)

4. mw01.vzwstatic (MidWest)

5. we01.vzwstatic (West)

The correct APN depends on where you are in the country. I did not find any more specific information than this and since Texas spans a few different regions I wasn't exactly sure which one I should use. I took a guess at so01.vzwstatic and it turned out to be the correct one. After setting this option the Cradlepoint 4G modem cap restarted and it grabbed the correct static IP from Verizon. Success!

If you switch back to dynamic IP's you should use the APN "vzwinternet" or use the default setting for the device. I found once my device was assigned static IP service and I restarted the modem I could not use vzwinternet. Seems like the APN has to match whatever Verizon has assigned on their backend or they will reject the device.

Monitor S3 file ages with Nagios

I have started using Amazon S3 storage for a for a couple different things like static image hosting and storing backups. My backup scripts tar and gzip files and then upload the tarball to S3. Since I don't have a central backup system to alert me of failed backups or to delete old backups I needed to handle those tasks manually. S3 has built in lifecycle settings which I do utilize but as with everything AWS it doesn't always work perfectly. As for alerting on failed backups I decided to handle that by watching the age of the files stored in S3 bucket. I ended up writing a Nagios plugin that can monitor both the minimum and maximum age of files stored in S3. In addition to monitoring the age of backup files I think this could also be useful in monitoring the age of files if you use an S3 bucket as a temporary storage area for batch processing. In this case old files would indicate a missed file or possibly a damaged file that couldn't be processed.

I wrote this my favorite new language Python and used the boto library to access S3. The check looks through every file stored in a bucket and checks the file's last_modified property against the supplied min and/or max. The check can be used for either min age, max age or both. You will need to create a .boto file in the home directory of the user executing the Nagios check with credentials that have at least read access to the S3 bucket.

The check_s3_file_age.py file is available on my github nagios-checks repository here: https://github.com/matt448/nagios-checks.

To use this with NRPE add an entry something like this:

command[check_s3_file_age]=/usr/lib/nagios/plugins/check_s3_file_age.py --bucketname myimportantdata --minfileage 24 --maxfileage 720

Here is output from --help:

./check_s3_file_age.py --help

usage: check_s3_file_age.py [-h] --bucketname BUCKETNAME
                            [--minfileage MINFILEAGE]
                            [--maxfileage MAXFILEAGE] [--listfiles] [--debug]

This script is a Nagios check that monitors the age of files that have been
backed up to an S3 bucket.

optional arguments:
  -h, --help            show this help message and exit
  --bucketname BUCKETNAME
                        Name of S3 bucket
  --minfileage MINFILEAGE
                        Minimum age for files in an S3 bucket in hours.
                        Default is 0 hours (disabled).
  --maxfileage MAXFILEAGE
                        Maximum age for files in an S3 bucket in hours.
                        Default is 0 hours (disabled).
  --listfiles           Enables listing of all files in bucket to stdout. Use
                        with caution!
  --debug               Enables debug output.

I am a better sys admin than I am a programmer so please let me know if you find bugs or see ways to improve the code. The best way to do this is to submit an issue on github.

Here is sample output in Nagios

Compiling libhid for Raspbian Linux on a Raspberry Pi

My son and I are working on a project using a Raspberry Pi and I needed to be able to talk to a USB HID device. This requires a software library called libhid but unfortunately it is not available as a package on Raspbian linux. I downloaded the source and attempted to compile it but ran into an error:

lshid.c:32:87: error: parameter ‘len’ set but not used [-Werror=unused-but-set-parameter]
cc1: all warnings being treated as errors
make[2]: *** [lshid.o] Error 1
make[2]: Leaving directory `/root/libhid-0.2.16/test'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/libhid-0.2.16'
make: *** [all] Error 2

After some googling I found a couple others on the Raspberry Pi forums that ran into the same problem. One of the commenters came up with a simple fix that requires a quick edit of the source code. In ~/libhid-0.2.16/test you need to edit the file lshid.c

Here is the code before making the edit:

39 /* only here to prevent the unused warning */
40 /* TODO remove */
41 len = *((unsigned long*)custom);
42
43 /* Obtain the device's full path */


Here is the code after the edit.
You need to comment out line 41 and then add len = len; and custom = custom;

39 /* only here to prevent the unused warning */
40 /* TODO remove */
41 //len = *((unsigned long*)custom);
42 len = len;
43 custom = custom;
44
45 /* Obtain the device's full path */

After editing the file simply run configure, make and make install like normal. The library will be put into /usr/local. Make sure you run sudo ldconfig before trying to compile any software that uses libhid. Thanks Raspberry Pi forums!

Bandwidth limits for guest wifi on an ASA 5505

At work we have free wifi for our customers as a nicety and so they can download our smartphone app if needed. Initially I set it up with no bandwidth limits with the idea of keeping an eye on it and locking it down if there was abuse. Over the past few weeks my MRTG graphs showed several spikes where the free wifi hit 10mbps. That is a big chunk of our internet connection so I decided it was time to limit the bandwidth. Since I'm not a Cisco expert it took some Googling to find the best way to do this. I found a couple resources that helped me put together what I needed. The free wifi network is on a separate VLAN with it's own IP subnet.

Here is interface definition for the VLAN

interface Vlan92
  nameif freewifi
  security-level 50
  ip address 192.168.92.1 255.255.255.0


Here is the syntax I used to limit the freewifi VLAN to 2mbps. The limit is applied to the subnet used by the freewifi VLAN.

access-list ip-qos extended permit ip 192.168.92.0 255.255.255.0 any

access-list ip-qos extended permit ip any 192.168.92.0 255.255.255.0

class-map qos

  description qos policy

  match access-list ip-qos

policy-map qos

  class qos

    police output 2000000 2000000

    police input 2000000 2000000

service-policy qos interface freewifi

Testing

My initial thought for testing the bandwidth limits was to connect to the freewifi VLAN and simply use one of the internet speed testing web sites. The speed test web sites worked fine for download speeds but the upload tests kept reporting that they were getting the full bandwidth of the connection. It seemed like the upload limit wasn't being enforced. I tried all of the popular speed testing sites and got the same result. Downloads were limited to 2mbps and uploads were running at the full speed of the connection. Hmmm...

I reviewed my settings on the ASA and everything seemed like it was correct. I decided to do a different type of test to see if I would get a different result. I created a 10MB file and then tested uploading and downloading it to and from a server out on the internet using scp. This test gave me the results I was expecting. Both upload and download of this test file took about 35 seconds which is inline for a 2mbps connection. I then tested transferring the same file on the inside VLAN which has no bandwidth limits and the scp transfer time was 4 seconds. I'm not sure what was going on with the speed test sites but the upload speeds were not reporting accurately for me.


Monitoring status

You can watch the bandwidth limits in action using the 'show service-policy police' command. If the limit is exceeded the output will show the number of packets and bytes that have exceeded the bandwidth limit.

This is the command output before sending any traffic:

asa5505# show service-policy police

Interface freewifi:
  Service-policy: qos
    Class-map: qos
      Output police Interface freewifi:
        cir 2000000 bps, bc 2000000 bytes
        conformed 1306 packets, 907993 bytes; actions:  transmit
        exceeded 0 packets, 0 bytes; actions:  drop
        conformed 0 bps, exceed 0 bps
      Input police Interface freewifi:
        cir 2000000 bps, bc 2000000 bytes
        conformed 1072 packets, 192021 bytes; actions:  transmit
        exceeded 0 packets, 0 bytes; actions:  drop
        conformed 0 bps, exceed 0 bps


This is the output after transmitting several test files:

asa5505# show service-policy police

Interface freewifi:
  Service-policy: qos
    Class-map: qos
      Output police Interface freewifi:
        cir 2000000 bps, bc 2000000 bytes
        conformed 149813 packets, 127878453 bytes; actions:  transmit
        exceeded 10273 packets, 14716462 bytes; actions:  drop
        conformed 3384 bps, exceed 360 bps
      Input police Interface freewifi:
        cir 2000000 bps, bc 2000000 bytes
        conformed 157493 packets, 123699017 bytes; actions:  transmit
        exceeded 15083 packets, 21214456 bytes; actions:  drop
        conformed 4928 bps, exceed 760 bps





Resources

Source: http://community.spiceworks.com/topic/137645-bandwidth-limit-vlan-on-asa-5505

Source: http://www.youtube.com/watch?v=Xb2A3Xekp7Q

Template Nagios check for a JSON web service

I wrote two different custom Nagios checks for work last week and realized I could make a useful template out of them. After writing the first check I was able to reuse most of the code for the second check. The only changes I had to make had to do with the data returned. So I decided to make this into a generic template that I can reuse in the future. The check first verifies that the web service is responding correctly and then checks various data returned in JSON format.  While writing this template I found this really cool service (www.jsontest.com) that let me code against a service available to anyone who wants to try out this Nagios check before customizing it. This is the first time I have used Python's argparse function and I have to say it is fantastic. It makes adding command line arguments very easy and the result is professional looking.

My github repo can be found here: https://github.com/matt448/nagios-checks

Here is the code in a gist:

Nagios file paths on Ubuntu and simple backup script

This is more of a note to myself than anything else but might be helpful to others. Here are the config and data directories for Nagios when installed using packages on Ubuntu 12.04.


Config files
----------------------
/etc/nagios3/
/etc/nagios3/conf.d
/etc/nagios-plugins/config
/etc/nagios

Plugin executables
---------------------
/usr/lib/nagios/plugins

Graphing (pnp4naigos)
----------------------
/usr/share/pnp4nagios/html
/var/lib/pnp4nagios/perfdata

Other
-----------------------
/var/lib/nagios
/var/lib/nagios3



Here is a very simple backup script for Nagios on Ubuntu 12.04

Relaying Postfix through AuthSMTP on an alternate port

AuthSMTP is an authenticated SMTP relay service that you can use with web applications or any situation you need to send outbound e-mail. Because it is an authenticated service it is a little trickier to configure Postfix to relay through their service. I found this post really helpful in configuring the sasl options but one thing I couldn't find a clear answer on was how to use a port other than 25 for the relay host. AuthSMTP offers alternative ports (23, 26, 2525) for SMTP because some ISP's block port 25. To use an alternative port just put a colon after the host name and add the port number. Like this (in main.cf):

relayhost = mail.authsmtp.com:2525


The entry in your sasl-passwords file must match the relayhost name like this:

mail.authsmtp.com:2525 username:secretpassword


Just a quick tip about something that wasn't obvious to me and hopefully this helps out someone else.