Category Archives: Avaya Stuff You Should Know

How to Parse Avaya Communication Manager data to a database for analysis – Part 3

Hello all!

This is a continuation of our discussion about pushing Avaya Communication Manager data into a database so you can analyze the results.

Here is part 1 where we connect to the Communication Manager through a perl script

Here is part 2 where we capture a bunch of data and parse it into useful tables

In this part, we push the data into a MySql database. The first step is to create the database. I typically use my own Linux workstation. You can ask your VM team for one. They take almost no resources and they’re open-source, so there are no licensing concerns. Therefore I typically have complete control of the repositories, database, and Apache instance. If you do not have the root password to your Linux workstation, then you might need help from your team with some of these steps.

We need to start with some assumptions here. So let’s assume you have followed parts 1 and 2 of this series and you have MySQL installed, okay?

Step 1 – create the database

Log into MySQL. These commands will create the database and user for this project. This user is called ‘scraper’ with a password of ‘crafty1876’

mysql> create database avaya;
Query OK, 1 row affected (0.01 sec)

mysql> use avaya;
Database changed

mysql>create user scraper@localhost identified by 'crafty1876';
Query OK, 0 rows affected (0.04 sec)

mysql>grant all on avaya.* to scraper@localhost;
Query OK, 0 rows affected (0.00 sec)

mysql>exit

We exit the database and log back in with our new user. This ensures that the user works and has full access to the avaya database

[roger ~]# mysql -hlocalhost -uscraper -pcrafty1876 avaya

Now let’s create the tables we need. I always have a “drop table if exists” statement so I can tweak the table as much as I want and re-create it. This creates a keyvalues table and a display name table.

mysql> drop table if exists avaya_keyvalues;
Query OK, 0 rows affected, 1 warning (0.06 sec)

mysql]> create table avaya_keyvalues (
 id integer auto_increment primary key,
 active integer,
 key_name varchar(256),
 key_value varchar(256),
 record_status integer,
 created_on datetime,
 updated_on datetime,
 deactivated_on datetime
);
Query OK, 0 rows affected (0.11 sec)

mysql>drop table if exists avaya_display_name;
Query OK, 0 rows affected, 1 warning (0.06 sec)

mysql>create table avaya_display_name (
 station varchar(12) NOT NULL,
 name varchar(32),
 PRIMARY KEY (station)
);
Query OK, 0 rows affected (0.09 sec)

mysql>

Now we have a database and the empty tables we need. Since you have read part 1 and part 2 of this process, you should have a perl script generating your ‘keyvalue.txt’ file.

Step 2 – Push the keys and values into the database

We have a new perl script that takes the file of keys and values and puts it into the database. You can click here for a download the script. Just rename it to .pl. Here it is so you can see it. It is called keys2db.pl:

#!/usr/bin/perl

#provided as-is by in 2017 by RogerThePhoneGuy.com 
#This script reads a file of "key/value" pairs, and logs them to a database. If the value was changed or deleted, it logs that too
#meant to be used with keyval.php, which shows this data
#assumes there's a local mysql connection

use strict;
use DBI;

my $db;

sub db_open { 
	my $dsn = "DBI:mysql:avaya:localhost";
	my $user = "scraper";
	my $pass = "crafty1876";
	while (!($db)) {
		$db = DBI->connect($dsn, $user, $pass);
		sleep 1;
	}
}

sub db_close { 
	if($db) {
		$db->disconnect();
		sleep 1;
	}
	undef($db);
}

sub generic_replace { 
	my ($table, $fields, $values) = (@_);
	my $sql = "replace into $table ($fields) values ($values)";
	my $sth = $db->prepare($sql);
	if(!($sth->execute())) {
		print "WARNING - unable to replace with $sql\n";
	}
	$sth->finish();
}

sub generic_create { 
	my ($table, $fields, $values) = (@_);
	my $sql = "insert into $table ($fields) values ($values)";
	my $sth = $db->prepare($sql);
	if(!($sth->execute())) {
		print "WARNING - unable to insert with $sql\n";
	}
	$sth->finish();
}

sub generic_read {
	my ($select, $table, $where) = (@_);
	my @datarow;
	my $sql = "select $select from $table where $where limit 1";
	my $sth = $db->prepare($sql);
	if($sth->execute()) {
		@datarow = $sth->fetchrow_array();
	} else {
		print "unable to execute $sql\n";
	}
	$sth->finish();
	return $datarow[0];
}

sub generic_update {
	my ($table, $set, $where) = (@_);
	my $sql = "update $table set $set where $where";
	my $sth = $db->prepare($sql);
	if(!($sth->execute())) {
		print "WARNING - unable to update with $sql";
	}
	$sth->finish();
}

sub generic_delete {
	my ($table, $where) = (@_);
	my $sql = "delete from $table where $where";
	my $sth = $db->prepare($sql);
	if(!($sth->execute())) {
		print "WARNING - unable to update with $sql";
	}
	$sth->finish();
}

db_open();

my $counter = 0;
generic_update("avaya_keyvalues","record_status=9","active=1");
open SCRAPED, $ARGV[0];
while (my $line = ) {
	chomp($line);

	my ($key,$value) = split(/\t/,$line);
	my $id = generic_read("id", "avaya_keyvalues","active=1 and key_name='$key'") + 0;
	my $currvalue = generic_read("key_value", "avaya_keyvalues","active=1 and key_name='$key'");
	if($id > 0) {
		if($value ne $currvalue) {
			$value =~ s/'/\\'/g;
			generic_update("avaya_keyvalues","active=0,record_status=2,updated_on=now()","id=$id");
			my $fields = "id,active,key_name,key_value,record_status,created_on,updated_on,deactivated_on";
			my $values = "null,1,'$key','$value',1,now(),now(),null";
			generic_create("avaya_keyvalues", $fields, $values);	
		} else {
			generic_update("avaya_keyvalues","active=1,record_status=1","id=$id");
		}
	} else {
		$value =~ s/'/\\'/g;
		my $fields = "id,active,key_name,key_value,record_status,created_on,updated_on,deactivated_on";
		my $values = "null,1,'$key','$value',1,now(),now(),null";
		generic_create("avaya_keyvalues", $fields, $values);
	}
	if($key =~ /\.station\.(\d+)\.status/) {
		my $station = $1;
		my $name = '';
		if($value =~ /^(.+?) \(ip=/) {
			$name = $1;
		} elsif($value =~ /^(.+) \(unregistered/) {
			$name = $1;
		}
		$name =~ s/'/\\'/g;
		generic_replace("avaya_display_name","station,name","'$station','$name'");
	}
}
close SCRAPED;
generic_update("avaya_keyvalues","active=0,record_status=3,deactivated_on=now()","record_status=9");

my $create = "
drop table if exists avaya_keyvalues;
create table avaya_keyvalues (
	id integer auto_increment primary key,
	active integer,
	key_name varchar(256),
	key_value varchar(256),
	record_status integer,
	created_on datetime,
	updated_on datetime,
	deactivated_on datetime
);
";

#my @count = generic_read("count(*)","callrecord","1=1");
#print "count is " . $count[0] . "\n";
db_close();

This script is really basic. It just does these things:

  1. Line 88, It marks all records in the database as ‘old’
  2. Starting in line 90, It cycles through the keyvalues file and checks for records where the value has changed (line 97). It marks these as ‘changed’ and then inserts the new record. It marks the unchanged records as ‘unchanged’ (line 104).
  3. If the current value is not in the database, it inserts it (line 110)
  4. It also replaces the value of the display name in case we want to see all extensions and names later (line 121)
  5. Lastly, any record that was not touched through this loop is NOT in the text file. Does that make sense? So any record that is still marked ‘old’ in step 1 must have been deleted from the Avaya system. So we mark these records as ‘deleted’ (line 125).

When this script is done, we have a list of all active records in the database. We also have a list of old values and deleted values. This gives you a rolling history of all extensions in your database. I cannot tell you how handy it is to see a history of an extension. You can see when the IP address changes, when registration changes, when firmware changes, network region, etc. And of course when a station is deleted you still have a record of it. This is very handy.

For those of you interested in Perl, there are also a bunch of helpful database functions in here. I keep these in my back pocket and often copy/paste them to the next project.

To tie this new perl script into our ‘sanity.sh’ script from last time, we should now have this:

file=/home/roger/avaya/sanity/data/sanitycheck_`date +"%Y-%m-%d-%H-%M"`.txt
perl /home/roger/avaya/sanity/av.pl /home/ARES/randerson/avaya/sanity/la.pbx /home/roger/avaya/sanity/commands.txt >$file
perl /home/roger/avaya/sanity/sanity.pl $file > /home/roger/avaya/sanity/keyvalues.txt
perl /home/roger/avaya/sanity/keys2db.pl /home/roger/avaya/sanity/keyvalues.txt

At the end of this shell script, you should have a fresh copy of all stations in your PBX in the database! Ready for reading with a simple PHP script. That will be next time, and hopefully MUCH sooner than this post. Sorry for the delay.

Before I finish, I want to thank you for reading this. I recently “moved” from an Avaya environment to a Cisco environment, so I don’t have easy access to the Avaya environment anymore. I am using some copies of the scripts so if you find any syntax errors or something doesn’t work, please let me know and we’ll get it working.

Once I finish this series, I’ll show you a way to AUTOMATICALLY generate labels for the 96xx phones. That’s the most exciting thing I’ve been wanting to share with you!

Roger

 

How to parse Avaya Communication Manager output into useful csv or database tables

Hello all!

This is a follow-up to my post about extracting data from Avaya Communication Manager. In that post, I showed you how to extract any “list”, “display”, or “status” from Avaya Communication Manager to a text file. First, I would like to show you my bash shell script that extracts this data. And then I’ll show you my perl script that converts the output to a text file that you can open with Excel or import into a database.

What information should you pull from Communication Manager? Well, in my case, I have a text file called ‘commands.txt’ that contains these lines:

display time
list ips
list surv
list med
status cdr
status trunk 1
status trunk 2
status trunk 8
status trunk 11
status trunk 16
status trunk 21
status trunk 79
list station
list reg
status station 2291
status station 2292
status station 2293
status station 2294
list call-forwarding
list off s
display time

So those are the various pieces of information that I care about. Looking at that info, those are my active trunk groups, and I want to know the status of four particular extensions (those happen to be the digital ports of my fax server). Also, I have a port network, so I want the status of IPSIs. Anyway, think about all the stuff you care about when you log into your PBX in the morning. And stick them in that “commands.txt” file. I put a “display time” at the beginning and end so I can look at the file later and tell how long it took to run the file.

So we want to run these commands automatically. So let’s create a file called “sanity.sh” and put this in the file:

file=/home/roger/avaya/sanity/data/sanitycheck_`date +"%Y-%m-%d-%H-%M"`.txt
perl /home/roger/avaya/sanity/av.pl /home/ARES/randerson/avaya/sanity/la.pbx /home/roger/avaya/sanity/commands.txt >$file
perl /home/roger/avaya/sanity/sanity.pl $file

Once you save the file, make it executable with

chmod +x sanity.sh

Let’s go over this script line-by-line, okay?

  1. The first line creates a variable for the file that will contain the data. There’s a little linux trick to embed a timestamp in the filename.
  2. The second line runs the file av.pl that I shared with you in a previous post about this script. It’s magical. I cannot take all the credit, but I have modified it for our purposes.
  3. The third line runs a file that parses the output of all the commands. This is a very useful script and I can take full credit for this one. It generates a list of “key/value” pairs to a text file.

My thought process with this script is to create a unique key for every piece of useful data. For example, I want the name, IP address, firmware version, gatekeeper, and network region for every station. I also want the EC500 mapping. And I would like the port number for analog stations. Oh! I also want a sense of low or high priority (Is it bad if the data changes?) and I need to include a site identifier in case I have the same extension in multiple sites. That’s a lot of information to display. And, if I have 1000 stations, it’s too much to display all this information on separate lines. That’s like 7000 lines of data for my stations. Fine for machines to read, but it seemed like too much for humans. I want to put them into one (and in the case of ec500, two) lines of data. I do this by assigning a “Key name” for my stations like this:

Key Name Key Value
lo.la.station.8348.offpbx EC500=>2135552978, OPS=>8348
lo.la.station.8348.status Anderson, Roger (ip=10.10.60.138;reg=3;ver=3.260A;gk=10.2.86.180)

See how much data I squeezed into that? And note that very few stations will have off-pbx information. Most stations will just have that station.status key. The script called sanity.pl takes the raw output from the Communication Manager and converts it to these key/value pairs above. Here is a link to sanity.pl. You’ll need to rename it to a ‘.pl’ file. This should compile just fine. When you copy it to your linux server, try

perl -c sanity.pl

That -c just means to check the syntax and don’t run it. It hopefully says “syntax okay”. Actually, I’m sorry but you’ll need to edit the file and change line 45 to the path where you want your ‘keyvalues.txt’ file, which will contain all your data. This should use a command line param, but it doesn’t. Sorry.

You might consider looking in more detail at this file. It’s where the magic happens and it’s a good framework for parsing other data. If you’re interested, here is a summary of how it works:

  • Starting in line 56, we check the raw text file line-by-line looking for the header for each ‘list’ or ‘display’ or ‘status’ section. For example, when we see the text ‘REGISTERED IP STATIONS’, then we know we are expecting to see the result of a ‘list reg’ (by setting our variable of type to 7).
  • Then on line 183 we actually process the registration status of stations. If type=7, then we are in the ‘list reg’ mode and if a line contains registration information for a station, then we capture the data.
  • Each section has some customization, but you’ll notice that (almost) each section performs a ‘set_data’ of the keys and values appropriate for that section.
  • At the end, the trunk group and station summary is generated and the keys are reported alphabetically.

Here are some examples of what the script does:

  • The script parses ‘list station’ and assumes all stations are unregistered.
  • The script parses ‘list reg’ and fills in the registration status for all stations (anything not parsed in this section has a status of ‘unregistered’)
  • The script parses ‘status trunk xxx’ and simply counts the trunks that are in service (ignoring the “idle/in use” values. We just count trunk members in service
  • The script parses ‘status cdr’ and simply stores the percent full of the buffers
  • The script parses ‘list off s’ and stores the off pbx mappings for all stations
  • The script parses various critical fields for IPSIs, media gateways, and survivable servers
  • The script parses ‘list call-forwarding’ and generates a list of all stations that are call forwarded.

At this point, if you ran the bash script, you would have an awesome list of keys and values in a text file called keyvalues.txt. You probably don’t need to know anything about Perl to get this working for you. But if you do know a bit of Perl, you’ll be able to do amazing things with it.

There’s more!

This data is useless unless you’re looking at it. In a future post, I will show you how to schedule this script with cron, push the data into a database, read it with a simple ‘web site in a single file’ php script, filter the data (including historical values!), and get alerts when the values change.

And then I have something REALLY amazing to show you!

Thanks for reading, everyone! Feel free to contact me at roger@rogerthephoneguy.com or post a comment here. If you need help getting this code working, or you want to tweak it a bit, let me know!

Roger

How to automatically extract data from an Avaya Communication Manager

You might call this post “the secret of my success” as a PBX admin for Avaya Communication Manager. Avaya provides some utilities for their techs and business partners, but customers don’t really have a lot of tools to easily extract data. In this post, I’ll show you how to extract ANYTHING from Avaya from a command line so it’s suitable for scripting. In my case, I parse the data further and generate a list of all stations and report of changes to my PBX every 15 minutes. Have you ever wondered who used to have extension 4438? How about that phone for the intern over the summer named “Bodkin Van Horn” that you deleted but now he has been hired and you want to give him the same extension? How about the question “how long has this extension been unplugged”?

Back in mid 00s, I wrote some PHP integrate with CM using telnet. Since I didn’t know much about PHP’s ANSI support, I ended up writing my own ANSI parser of the raw data. Anyway, it was great, but many customers don’t support telnet and want all communication to CM to be through ssh, which is reasonable, right?

Then I found this post from Benjamin Roy where he provided a perl module to do it through ssh. Also, he meticulously reverse-engineered Avaya’s OSSI protocol, which I think is impressive and sounds like fun. But I didn’t use OSSI; I just wanted a way to get data using ssh. So I took Ben’s code and tweaked it a little bit. It’s straight perl code – it’s not a module or anything like that. It does have dependencies on a couple other modules though. You do not need to be a perl developer to use this. It’s really easy. Here is a link to the script. Just save this as “av.pl”. Try to run it with “perl -c av.pl” (the -c means just check to make sure it compiles but do not run it). You might get some errors about missing modules. There are only three, and this is how you install them if you need to:

perl -MCPAN -e "install Expect"
perl -MCPAN -e "install Data::Dumper"
perl -MCPAN -e "install Term::VT102"

I should probably mention here – if you don’t have a Linux server in your environment, you should really get one. Your Virtual Machine team can make one for you – just about any distribution of Lunux will do and they take almost no resources. If you need to justify it, just say something like “utility server for PBX monitoring” or something like that.

Anyway, back to the script. Naturally, it needs to connect to your PBX, so create a little text file like “mypbx.txt” with the contents

10.10.40.89,5022,cmusername,Passw0rd,ssh

Obviously, replace the IP address, username, and password of your PBX.

And then create another file called “commands.txt” with the contents

list trunk
display station 3100

This can contain whatever commands you’d like. list reg, list station, list trunk, list locations, list measurements, stat cdr, etc. It should be any command that can be “paged” through, you know?

And then simply run this line

perl av.pl mypbx.txt commands.txt

The script will connect to the PBX defined in mypbx.txt, run the commands in commands.txt and output the results to STDOUT!

This is the core script to a bunch of stuff I automate in my PBXs. In a later post, I’ll show you how I parse the results with Perl and make use of the data.

I call this “the secret of my success” because I use this data to build a simple html page of all extensions in all of my PBXs every 15 minutes. I keep this page up all the time in a browser tab, which enables me to simply “control-f” for any name in the PBX and I have the extension number and the IP address of the station (or ‘unregistered’ if it’s unplugged). I can see unregistered ‘guest’ numbers, see the station’s history, etc. There’s a lot more I do with this information and I look forward to showing you.

Please let me know what you think! And if there’s anything I can do to help you get it working.

Lastly, if any of you need to do any telephone system testing (capacity, QA, DID number ports, call-flow, queuing, etc), please check out my post about CallsByCloud. If you sign up and use the promotion code ‘roger2015’, you’ll get $10 in credit and a 20% discount on the per-minute rate.

Also, if you hate unsolicited outbound telemarketing, consider subscribing to the Jolly Roger Telephone Company. Your telemarketers will entertain and delight you. Sample recordings here.

Thanks all!

Roger

 

Avaya Call Detail Recording to Kiwi syslog server

In a previous post about busy/release of CDR, I asked if anyone was curious about capturing CDR from Avaya Communication Manager using a Kiwi Syslog server. At least one of you was, so here it is.

There are two parts of course. One part is programming the phone system to send CDR. And the other part is programming Kiwi to capture CDR. These can be done in either order, but if you program the phone system first, the CDR buffer will fill as is tries to establish the connection with Syslog.

So let’s start with Kiwi. Your version may differ. In my case, the network team already had a rather powerful installation of Kiwi on some great hardware. All I had to do was configure it to capture Avaya CDR. So when you launch the Kiwi dashboard, click File->Setup and in the “Server Setup” window, scroll to the bottom for “Inputs”.

kiwi-config-0

In the TCP section, tell Kiwi to listen for TCP messages on whatever port you wish. I’m using 5013 – probably because I saw it in an Avaya doc somewhere. But you can use any port.

Then scroll up to the “Rules” section and configure a new rule to match for messages from your Avaya Communication Manager.

kiwi-config-1

In my case, I named my rule “Call Records from LA PBX”. I set a filter by IP address. If you also send any other information to Kiwi from the CMs, then you’ll want to filter by the Priority field. You want the Local7 field to be Debug. I don’t recall why this works. I think I was looking at the data one day to try to separate the CDR from the other syslog messages from Avaya and this seemed the easiest way to do it.

kiwi-config-3

Once you have those two filters – IP address and Local7.Debug, you need to create an action. My action above is a little misleading. All you really need is one that logs to a file, then stops processing. If you’d like, you can also display it to one of the virtual displays in Kiwi. I log to a text file on the local file system:

kiwi-config-2

Now, if we program our PBX correctly, we should get CDR to these text files!

Step 2 – Program the Avaya Communication Manager

Programming the Communication Manager is three steps: Add the node-name, set the ip-services, and change the system-parameters cdr.

To set the node name, ‘change node-name ip’ and create an entry for the IP address of your Kiwi server. This is basically the DNS lookup for the CM. This sets the name and IP address. Any name will do. Enter the IP address of your Kiwi server.

change node-names ip                                            Page   1 of   2
                                  IP NODE NAMES
    Name              IP Address
SL_LSP              10.10.60.10
CHA_LSP             10.10.1.10
Chicago             10.10.148.2
Chicago_LSP         10.30.10.10
DO_LSP              10.20.16.10
Smoot               10.87.176.11
Gateway006          10.46.12.1
LA                  10.10.19.6
LA_ESS              10.44.192.10
LE_ESS              10.58.200.10
LA_SM               10.59.29.16
( 16 of 44   administered node-names were displayed )
Use 'list node-names' command to see all the administered node-names
Use 'change node-names ip xxx' to change a node-name 'xxx' or add a node-name

Next, set the ip-services. I have squeezed all three screens here. The important thing here is page three. We need to set the “Reliable Protocol” to n. I think Avaya defaults this to y and it works great for their pre-packaged CDR partners. For home-grown syslog, we set this to n. It may not hold up in court if there’s ever a dispute.

change ip-services                                              Page   1 of   3

                                   IP SERVICES
 Service     Enabled     Local        Local       Remote      Remote
  Type                   Node         Port        Node        Port
CDR1                 procr            0       la_syslog       5013
CDR2                 procr            0       prognosis       50000

change ip-services                                              Page   2 of   3

                                   IP SERVICES
 Service     Enabled     Local        Local       Remote      Remote
  Type                   Node         Port        Node        Port


change ip-services                                              Page   3 of   3

                              SESSION LAYER TIMERS
  Service     Reliable  Packet Resp   Session Connect  SPDU  Connectivity
   Type       Protocol     Timer       Message Cntr    Cntr     Timer

 CDR1            n         10                3          3         10
 CDR2            n         10                3          3         10


And last step – set the system-parameters for CDR. Here’s the thing. You have lots of options here. You can define fields, separators, etc. But I discovered that there is a line length limit that is not obvious. Also, the caller-id name is surprisingly absent. Rather than customize the CDR, I have gone back to “unformatted” and let further processing figure it out.

change system-parameters cdr                                    Page   1 of   1
                            CDR SYSTEM PARAMETERS

 Node Number (Local PBX ID): 2                     CDR Date Format: month/day
      Primary Output Format: unformatted   Primary Output Endpoint: CDR1
    Secondary Output Format: unformatted Secondary Output Endpoint: CDR2
           Use ISDN Layouts? n                   Enable CDR Storage on Disk? n
       Use Enhanced Formats? n      Condition Code 'T' For Redirected Calls? n
      Use Legacy CDR Formats? y                 Remove # From Called Number? n
Modified Circuit ID Display? n                             Intra-switch CDR? y
                  Record Outgoing Calls Only? n     Outg Trk Call Splitting? y
  Suppress CDR for Ineffective Call Attempts? n       Outg Attd Call Record? y
      Disconnect Information in Place of FRL? y      Interworking Feat-flag? n
 Force Entry of Acct Code for Calls Marked on Toll Analysis Form? n
                                    Calls to Hunt Group - Record: member-ext
Record Called Vector Directory Number Instead of Group or Member? n

     Inc Trk Call Splitting? n
  Record Non-Call-Assoc TSC? n           Call Record Handling Option: warning
      Record Call-Assoc TSC? n   Digits to Record for Outgoing Calls: dialed
   Privacy - Digits to Hide: 0               CDR Account Code Length: 15
Remove '+' from SIP Numbers? y


I have a perl script that parses unformatted cdr. Anyone want to see it?

Anyway, when you have this set up, you can status CDR to make sure the link is up:

status cdr-link
                                CDR LINK STATUS
                   Primary                      Secondary

       Link State: up                           up

      Date & Time: 2016/04/17 08:43:29          2016/05/09 10:12:20
  Forward Seq. No: 0                            0
 Backward Seq. No: 0                            0
CDR Buffer % Full:   0.00                         0.00
      Reason Code: OK                           OK





The good news here is the link state(s) are “up”  and the CDR buffer % full is 0. If you have trouble, you can “busy cdr primary” and “release cdr primary” to close and open the socket. You should see a blip in the Kiwi server when this happens.

Make a call and watch Kiwi! You may need to check the log files rather than rely on the Kiwi display. The extra CR/LF will sometimes make Kiwi look funky but the text file is usually just fine.

Let me know how it goes for you!

Roger

 

Avaya CM – will I lose the CDR buffer if I busy/release the cdr link?

I have an Avaya Communication Manager sending CDR to a kiwi syslog server. For some reason, the link is down and the buffer is filling up. What happens if I busy/release the CDR link? Will I lose the buffer? Scary!

Short answer is no. I just tried it. I performed a busy/release on the primary CDR link and the buffer is still at 40%. Unfortunately, the link is still down so I’ll have to check with the syslog team. But I just wanted to let all of you know that it’s safe to restart the link.

status cdr-link
                                CDR LINK STATUS
                   Primary                      Secondary

       Link State: down                         up
Number of Retries: 18
      Date & Time: 2015/09/23 06:35:59          2015/09/12 07:46:56
  Forward Seq. No: 0                            0
 Backward Seq. No: 0                            0
CDR Buffer % Full:  41.21                         0.00
      Reason Code: CDR connection is closed     OK


I ended up disabling and then enabling the TCP listener in Kiwi and the buffer immediately cleared. Is anyone interested in a post on how to capture CDR to a SYSLOG server?

Flowchart of Avaya Communication Manager Routing

Click Here for the flowchart PDFs

One might argue that telephone systems do only one thing. They route calls. In the process they turn lamps on and off, and they make telephones buzz and beep. But the only point is to route voices from point A to point B.

If you’ve been working with Avaya Communication Manager for a while, you have been exposed to AAR and ARS routing. And you hopefully have a fair grasp of how it all fits together. But there are a whole lot of decisions that are made when routing outbound calls. And if you’re new to Avaya, it might seem overwhelming to troubleshoot routing issues. So here are four pages of flowcharts that, based on my experience, are the most common configuration issues you’ll see when managing your telephone system. Click Here for the flowchart PDFs

Page 1 is a flowchart of the digit collection. This is what the CM does to collect the dialed digits, translate them as necessary, and select a network (AAR vs. ARS). Digits can be translated many times and re-pointed to different networks. Once this process is complete, the logic continues to page 2.

avaya_call_routing_flowchart-page_1_of_4

Page 2 is the process of using the final dialed digits (and station’s location) to select a route and a trunk group. There are many location and facilities restriction level checks here. And there is even more digit translation that can take place during this process. Once a trunk is selected and the call delivered, this may not be the end of it. Look Ahead Routing may pull the call back and try again.

avaya_call_routing_flowchart-page_2_of_4

Those two pages compose the primary routing logic that you will use in your typical troubleshooting. However, many decisions are based upon the calling party’s location. It may be obvious to you how the CM know’s the station’s location. But if things get strange, you can see how the CM associates a station to a location on page 3.

avaya_call_routing_flowchart-page_3_of_4

And lastly, there is one major headache associated with the “Prefix Mark” in a route pattern. Prior to 1995, there were many, many dialing rules associated with seven digit dialing, one plus seven digits, ten digits, etc. I remember living in a small town in the California Bay Area, and to dial Concord (over the hill), I had to dial 1+seven digits. Then when the area code 415 was split in to 415/510, we could dial 10 digits between them. Unfortunately, if you dialed 1+10 digits, it would be charged as a long distance call. All this went away in 1995 when the North America Numbering Plan was implemented. It is much easier to route calls now. But there is still the dreaded “Prefix Mark” that may still be in your routes. This causes havoc with the dialed-digits that are sent to your carrier. Page 4 is a summary of how this prefix mark works.

avaya_call_routing_flowchart-page_4_of_4

Please note there are two extremely important troubleshooting tools when dealing with Avaya routing. One, is the ‘list ars route-chosen’ and ‘list aar route chosen’ commands, and the other is the ‘list trace station’ and ‘list trace tac’ commands. If you use Avaya Site Administrator, you will not be able to use the ‘list trace’ command. I encourage you to use telnet or ssh in your day-to-day management of your PBX so you can use these commands when necessary. It may not sound like much, but being able to list trace at a moment’s notice is very helpful.

I will save the “list aar/ars route-chosen” and “list trace station/tac” for another posting. There’s a lot of information available in these commands.

Please let me know if you find these flowcharts helpful. Also, if you find any typos please let me know. Please keep in mind there are many more routing decisions the CM makes that are not in these flowcharts. If your CM uses other routing (for example tenants or time-of-day) and you think I should incorporate them into these flowcharts, please let me know! Happy routing everyone. This is really the core of any PBX, so I personally really enjoy working with routing.

Roger

Click Here for the flowchart PDFs

How to reboot the Avaya S8300D from the G-Series gateway

I recently had a problem logging into System Platform. I could not connect to the CDOM via the web, nor SSH into DOM0. Fortunately, there is a way to reboot the S8300D from the gateway itself. Log into the gateway as root, then type “configure” to get to the config menu. Then simply type “reset mm v1”, which will reboot the media module in slot V1. Confirm, and the card will reboot. From my understanding, it will cleanly shut down each VM and reboot.

As always, it will take just long enough for you to panic, plus 45 seconds or so.

 

TraceSM is already running – how to stop it

In the new version of Session Manager (6.3 and above), the traceSM process is supposed to timeout and stop on its own. However, I was just trying to run it today and got this error

[cust@la-sessionmanager ~]$ traceSM
ERROR: traceSM is already running. Only one instance is allowed.

Uh oh. I don’t know the root password so I cannot kill it. However, Avaya thought of this for me. Just run traceSM with the -k switch and all is well:

traceSM -k

This kills the old process and runs a new instance for me. Just another thing that sounds simple once you know it.

How to clear PLAT-ALM alarms in Avaya Communication Manager

When gateways unregister due to various network blips or maintenance, it can cause a PLAT-ALM in your Communication Manager. These will linger forever until you clear them, It’s very simple but you don’t do it from SAT, you do it right from the shell. To see all of the alarms, login to CM’s shell and type

almdisplay -v

To clear them all, just type

almclear -a

If you still have a problem, the alarm will re-propagate so I haven’t seen the downside.

How to remove an extension when there are messages waiting

Just a quick note regarding the message

Message(s) waiting; please delete message(s) first

If you try to remove a station with messages waiting, the Communication Manager will reject it if there are messages waiting. In my case, I wanted to convert a virtual station to a physical 9630, but the system won’t let me remove the station. And of course, I cannot change the type of station from Virtual to 9630 – I get this message:

"9630" Station must be removed and re-added to change type

As usual, the solution is simple once you know it. You can clear the message waiting indicator with “clear amw all xxxx“. This clears the messages flag (not the messages!) and turns off the message waiting indicator. This allows you to remove the station.