atcomsystems.ca/forum Forums TDM Business Telephone Systems Avaya - Lucent Telephones & Systems 400x series IPO: AMI vs B8ZS in VoIP environment

Hi All, I am new here and found this site looking for info on how one AMI/SF T1 would behave in a VoIP environment where all other T1s are B8ZS. We have an Avaya IPO 400x series and 4 locations connected by T1s. We have had a problem with dropped calls and I believe that the AMI circuit is the cause due to the fact that all calls that come into the location with the AMI circuis and are transferred to a spoke location drop 100% of the time. This doesn't occur when the call is transferred to the hub site, even if the hub then transfers to another spoke. I am having the AMI circuit replaced with a B8ZS circuit on Monday 3/31, and I will have my answer at that time, but I am documenting the issue as we have had this problem for nearly a year and my BP said that the AMI circuit shouldn't be causing problems. Can anyone point me to a white paper or some other documentation that will show that the mix of linecoding can cause serious problems?

If the circuit is designed for and optioned AMI all the way through it should work find. If your equipment calls for B8ZS and you have AMI than it won't. That simple.

Hi justbill. The AMI circuit was installed years ago, before I came to work here, and was intended for use with non-voice data. I am not certain that the Avaya equipment has a specification for B8ZS, but given my BPs overall incompetence, I have no intention of asking, I can find out. The BP didn't even get the original config right, I had a problem for the first 2 months where you would try to call or transfer to a spoke and get and "incompatible" message on the phone screen. The BP spent those 2 months blaming my network and not looking at their config. I found an Avaya forum and they had it solved in one day. So I have no faith in the BP. I was thrown into this by default, and we have combed the network and the IPO configs to the point of absurdity. We've upgraded routers and switches, implemented QoS, combed over physical wiring, all to no avail. We use very little bandwidth overall, so no congestion, no jitter, nearly no packet loss, etc. I have seen several things that indicate that if the signal traverses a B8ZS circuit, then hits an AMI circuit, and then goes to another B8ZS, that this can cause problems. The moment of truth is 10AM Monday morning. IF it solves the problem, I have a little gloating to do over the BP. If it doesn't, well...

I am no VoIP expert, by any stretch of the imagination. I do understand T-1's. If you come out of a switch optioned either it has to be the same on the other end, you can't change in mid stream. You can go (Network wise) from a B8ZS optioned switch into another B8Zs optioned switch out of the switch AMI to another optioned AMI and out B8ZS. It's what happens end to end that matters, what goes on internal to the switch really doesn't matter. Again, I'm not talking Telco to a PBX I'm talking Telco to Telco. So I'm not sure what you mean by going from a B8ZS circuit to an AMI circuit. If it's internal to the switch it doesn't matter.

We've got some good folks here that maybe can make more sense of what I'm trying to convey.

justbill, I am probably not making myself very clear on that. The calls come into the PRI, which is B8ZS, the IPO forwards the call based on the call route for the number called. In this case if the caller calls the number specific to the spoke named Hwy, which has an AMI circuit, then the voice data is going from the B8ZS PRI across the AMI circuit to the Hwy. No problem there. But if the caller wants a different spoke than the one they actually called, and the Hwy transfers to that spoke directly, the traffic is then going back to a B8ZS circuit, and the call will drop 100% of the time. If I am not mistaken, that means that it goes from B8ZS to AMI then back to B8ZS. Incidentally, if the Hwy transfers the call back to the hub, and then the hub transfers to the requested spoke, the calls don't drop. This is the workaround we are using for now. I hope that makes sense and if I'm wrong hopefully someone will correct me

Thanks, Dawn

That’s not an entirely true statement… I’ve have little experience with VoIP but in the TMD world, it’s not uncommon for carriers to be “back to backed” with AMI & B8ZS carrier facilities.

For your situation, if all is configured/optioned correctly there should be no “issues.” The only caveat to this is if “clear channel,” 64 Kb, or “switched 56Kb” is in the mix via optioning for the traffic between offices. In which case the signal would never survived on the AMI carrier to start with. If that was introduced to your “spoke” location I could see how a call could be immediately dropped.

That made me dizzy…

Honestly, I don’t think your AMI T1 could or would be causing your dropped transferred calls problems. The T1 is just a pipe. It seems to me as you describe it your AMI T1 is capable of handling the signaling and voice traffic in and out of the Hwy spoke in all other cases. I don’t see where it would be intruding a problem on transfers back to “hub.”

Well I guess you could say I'm grasping at straws at this point. This has been going on for almost a year and I am desperate to fix it. And I should tell you that the calls don't drop immediately, they stay connected from 30 seconds up to 2 minutes. I have had considerable help from an Avaya forum but we just can't find the problem. Interestingly though we have another spoke that was on AMI and appears to have had the same problem, which appears to have been resolved by conversion to b8zs. I say appears because I hadn't refined the circumstances of the drops until after I had had it converted. I am kicking myself for it now though.

I should also say that there is a huge amount of detail that I haven't posted here. One important thing though is that there is always an error with a drop, and it is always the same. I'm on my phone right now, but when I get home I will post that error and what led me to this point.

Here is my persistent error (call information removed):

176379090mS ERR: EXCEPTION ON MEDIA CONNECTION --- Error from protocol entity! MSDSE --- local timer expired ...[call information]…
176379091mS CD: CALL: 40.64.1 Deleted
176379095mS CMLineTx: v=40
CMReleaseComp
Line: type=IPLine 40 Call: lid=40 id=64 in=1
Cause=2, No route to specific transit network/(5ESS)Calling party off hold

What led me to this point was looking at the call origin rather than the call destination (our orignal heavy focus). Without exception, the dropped calls originate at an AMI spoke and are transferrred directly to a B8ZS spoke, bypassing the hub; it always delivers the error above. We have funny little clusters of the same caller being dropped over and over, and that is obviously because they were always calling back to the same number, and always being directly transferred to the same destination. By transferring the call the to hub first, then transferring it to a spoke destination, the calls remian connected, otherise they all drop in that 30 second to 2 minute timeframe. My study of the error (and there is much more if you look at the whole call from origin) shows me that there are master and slave entities which have to send acks back and forth for the call to be viable, and it appears that somhow that process is breaking down and an ack is not being passed in a timely fasion (I think, this is outside my field of expertise). Attempts to learn more about the second portion of the error, the "no route to specific transit network", have been fruitless. I am prepared to do packet analysis if the new circuit does not solve the problem, but I am unpracticed at that particular skill.

Dawn

Thank you for providing us with the IPO output error message with your other important information on your symptoms. It seems that all symptoms and errors are isolated to only calls associated with your sole AMI/SF T1. If this is true, let's focus on two potential causes for the dropped calls and error messages:

1. A software programming error within the IPO, which is causing the calls being processed from the AMI/SF circuit to drop when bypassing the hub.

OR

2. Some possible rough history between AMI coded and SF framed T1 circuits and the Avaya IPO 400x series systems. This may require considering having that circuit's coding and framing updated to B8ZS/ESF to match all of your other T1s.

Like others here, I'm not an expert with VoIP or the Avaya IPOs. However, I suggest that we consider moving this thread over to our Avaya forum to get some additional input which would be more specific to your system (to include updating the title of this topic to reference the 400x series IPO).

Just let us know, and one of us would be happy to move this discussion over to Avaya. :thumb:

Dawn -

Could it just be the T-1 "Pipe" that's causing the trouble and not the line coding? Have you stress tested the AMI "T"?

If stress testing doesn't show any problems, than by all means, try to change the coding (and then retest to make sure the circuit is good in its new configuration!).


Sam

Hi guys and thanks for the welcome and assistance. I have no problem with the thread being moved over to Avaya, whatever works! In fact the AMI circuit is being disconnected and cutover to a B8ZS circuit Monday 3/31 at 10AM CDT. They are putting us on a new pair, not just moving the existing pair, so I am very hopeful.

We never did stress test the AMI circuit, just the PRI and the T1 to the site where most drops occur (numerous times), we had the occasional slip, but nothing that corresponded with drops. The reason for this was that we were never looking at the origin. Please understand that I was being led around by a BP who I have discovered is less talented than I am (not boasting, I am a rank novice in voice telephony), and it wasn't until I decided to abandon everything the BP had to say that I started focusing on the call origin, and wishing I'd done so half a year ago. Once the cutover is complete, it will take about 5 minutes to determine if the change made a difference (gotta love a 100% drop rate, it'll make it easy to determine if this worked).

-Moved to Avaya, initial posting member notified

Thanks! Dawn

Well, it was worth a try, but the drops remain after the circuit conversion to B8ZS. If anyone has any other ideas, I am all eyes...

Thanks, Dawn

Are the call failures still isolated to only calls from this "B8ZS-updated" circuit that attempt to "automatically bypass the hub site"?

With today's conversion to B8ZS coding:

1. Was the T1's framing also updated from from SF to ESF?

2. Do calls from this updated circuit that are transferred TO the hub site still process OK?

5Etek-mike, yes, the framing was also updated, I did the routers on both ends. And yes, the calls transferred to the Hub still process just fine. The situation is unchanged.

Meanwhile, here is a little more info (the Hwy & 21 references are both spokes):
I was watching system status monitor during test calls, and I hope this makes sense:

If I call the Hub, and have them transfer me to the 21 (the ususal destination for drops) I don't drop, and I can see the call in both status monitors, the Main and the 21mm.

If I call the Hwy (the originating source for all drops) and have them transfer me to the 21, not only will the call drop after about 1:30, but the call vanishes from the Hwy status screen as soon as it is picked up at the 21. It does this if transferred to the Hub too, but stays connected.

If I call the 21 and have them transfer me to either the Hwy or the Main, I can see the call in both status screens throughout the length of the call. Only at the Hwy does the call vanish upon pickup no matter where the call goes.

I might also add that if an internal user calls from the Hwy to another internal user at the 21, the call won't drop. They are incoming calls only.

Hmm Mike asked me to join in on this thread. I seem to remember some of your nightmares on another forum if this is the same Dawn.

As bad of a BP you obviously have/had, he was correct that the line coding etc should have nothing to do with your dropped calls just as you've noticed with the lack of change with the new circuit.

The problems you are having seem to be pointing back to the IP Office as the culprit. Since I haven't followed all of the previous threads please fill in some "blanks".

If I understand the situation properly, all calls come from a PRI in the "hub" then, are routed via IP lines to all of the "spokes"?

So a call is routing from a PRI on one system to another via IP line to a second system, then transfered to a third system that is connected to the original system via a different IP line.??

Another helpfull tidbit is give us a run down of the systems in question. Model/version/etc.

If this is the case, how many IP trunks are configured the different sites? How many systems are connected? Is this an scn? And, as IPOfficeguy asked, what releases are we talking?

ipofficeguy, I think I am probably the same Dawn if you are referring to the tek-tips forum, I still have posts there too <g>. Ok, here we go...

Yes, all calls come in to the hub on a PRI and are then routed over IP lines to the destination indicated by the called number, including all spokes. When a call comes in for the Hwy spoke, it rings there, but I am assuming that it comes into the PRI at the hub before heading to the Hwy. Then if the call is for aother spoke, is would be sent to that spoke directly. After approx 1:30 the call will drop.

There are 3 IP trunks configured at the hub (numbered 10,11,12), one IP line at each spoke (numbered 20,30,40). It is an SCN, we are using IP406 Office, IP400 Digital stations, and at all but one very small site IP400 Phone units. The versions are:
Hwy: 4.0(14) & 6.0(14)
1mm: 4.0(14) & 6.0(14)
21mm: 4.0(10) & 6.0(14)
3mm (hub): 4.0(14) & 6.0(14)

I think that the 21mm is out of date because over the weekend we had a component fail and it had to be replaced yesteday (they lost all phone service there). The BP didn't mention needing to upgrade it, but I am going to assume that I will need to do that.

Hey, it looks like there is updated software for all units, admin 4.1(12)? Do I need the user 4.1(17) release and the VM Pro 4.1(40) release too?

Also, do I need to do the maint releases from December 07 first? And is there a way for me to be notified when these releases come out?

In those IP trunks that are configured (one trunk for each remote system) how many channels are there for each of them?

Also, what type of connection is the IP trunk using? Free internet? Point to point T1?

mongo5150, I have them configured as 4, 6, & 6. Originally they were configured for 4, 8, & 8, but we only have a total of 16 VCMs at the hub, and if I understand correctly, the total of the spokes can't exceed the total at the hub. Please clarify if I am incorrect.

They are connected by the PTP T1s, I have no internet traffic at all crossing them, internal data and voice data only, with class-based QoS on the routers to give the voice traffic 50% dedicated bandwidth. Before implemeting the QoS, if I flooded a T1 with data and placed a call to that location, it was obvious in the horrid quality of the call. Since implementing QoS that is no longer a problem.

Just to make sure I understand this. The PRI where all calls come in is at the main (HWY) site?

Then if the call is meant for the spoke site, it routes there as per the incoming call route...

Where the problem is, is after the spoke answers or whatever, when the call gets rerouted back to the main site, that is when the problem occurs?

Do these site all have their own voicemail, or is this centralized?

Was there a network assessment done? IF so, were the QOS settings made afterwards?

Simple answers first..
If you are going to upgrade to 4.1.++ then yes you would also need to upgrade the user and VMpro release. You don't need the maint release from Dec, and there isn't an easy way to be notified of new releases other than to check the site every 3-4 months. (updates are scheduled to be released quarterly....but that isn't always the case)
You can have more channels and/or ip lines configured than what you physically have in the system, just as you generally have more phones than lines, but you would only be able to utilized the amount of channels you actually have.

You might get lucky and resolve the issue you are having by upgrading to at least the latest maint. release.

The only other obvious thing I can think of at the moment would possibly be "direct media path" being checked when it shouldn't.

It really is a strange setup with an even stranger problem. I haven't noticed a bug listed with this type of problem but since I haven't encountered this setup myself yet, I haven't been looking for it either but will check the caveat list for something similar.

Have your business partner check into these...


CQ39306

Call between remote site drops after 90 seconds on ISDN line

4.0.10

MT_RELEASE_2Q08_4.1


3-Available

6.0.102804


CQ58088

External calls into PRI forwarded to remote SCN site drops after 90 seconds.

4.0.7

MT_RELEASE_2Q08_4.1


4-Proven

4.0.75060

mongo5150, no, the main site is NOT the HWY, HWY is a spoke. Only calls that originate at the HWY (I am assuming that they really originate at the HUb on the PRI but go directly to HWY via forward not incoming call route), and are transferred to another spoke will drop. If the call comes into HWY and is transferred to HUB before being transferred to a spoke, the call will stay connected. Voice mail is centralized. And NO, the BP did NOT do a NA, though it is a requirement. I had absolutely zero experience in phone systems when this was installed, we relied on the BP to tell us what we needed. He didn't. We have been trying to get him to do one ever since. He won't. I have acquired the knowledge I have of Avaya because of places like this where people with much more knowledge than I are willing to share their expertise. My background is in PCs and LAN/WANs with non-voice data, or I'd really be lost.

ipofficeguy, LOL! You already know my BP is, well, shall we say less than helpful? At this point, the BP and I only speak when forced to do so. I was in fact pointed to the docs you mentioned over at tek-tips, and indeed I plan to upgrade and hope to get lucky :)Thanks for the advice on what I need and what I don't. None of my units have "direct media path" enabled, we took that out sometime last summer. It was only when I stopped paying attention to the BP that I got THIS far. The BP never looked at where the call came from, only where it went, as it turns out, where it started is key, or we wouldn't have the workaround (that I discovered all by myself <g>).

I figured it wasn't a silly thing like direct media path and seem to remember that being discussed earlier.

I also know that your BP is less than helpful but had to suggest it since this truly sounds like a bug in the firmware. According to the list, it should be corrected in the 4.1 versions if the error listed is the same as yours. If it isn't, you have a few choices. Listed in order of cheapest, to most expensive:

Keep dealing with the problem as you have been,
Hope someone else has the same problem and it gets reported to Avaya for a fix in the next maint release,
Find a new business partner that will help,
Kick your existing BP in the head until they help,
Pay outrageous amounts for direct Avaya support,
Buy a bunch of hardware to convert the P2P T1's to data+TDM based tie lines,
Or scrap it all.

Please keep us posted with the results.

ipofficeguy, I like your suggestions I will keep the thread going, don't worry! I am planning on doing the upgrade within the next couple of days. My boss insisted on asking the BP who was supposed to be doing them (we're still under warranty, and this is the same BP who didn't want me in the system because "I might mess it up"). Personally, I'm ready to do the upgrade right now! I'm just dying to solve this.

Now, I will add my 2 cents worth, if your upgrade and new T-1's don't fix the problem. I had a similar situation happen awhile ago, where the site said they kept dropping calls. We change feeder pairs on the T. Stress tested the T, but everything came back to the IP Office. I also swapped out the PRI card.

What wound up fixing the problem was running a "new" Cat 5E cable for the T-1. If looked at on a t-bird everything looked fine.

The new cable was put in, T-1 moved over to it, everything has been fine ever since.

I know it sounds strange, but this did work for me when everything didn't.

doubled, thanks, that is an easy one! I am doing the upgrade tonight and if that doesn't fix the issue, I'll give that a try.

Can anyone tell me where I can find Avaya patches? I know where to get the maint releases, but the BP is telling us that there was a patch that was released the 28th of March, and all I see is the March 4.1(12)? He didn't provide me with a link.

Also, I appear to be a non-person again over at tek-tips, ipofficeguy, if you have a chance, please drop in on my active threads and let the people over there know I'm not ignoring them, but I can't logon and tek-tips NEVER answers me, this is the third time and I have no idea why.

but I can't logon and tek-tips NEVER answers me, this is the third time and I have no idea why.

Why would you bother with them when we have the best people here?

-Hal

Since you are on 4.0.14 on most of your systems already, 4.1.12 is the next higher version. The only other 4.0.xxx patches out there are the private builds that you wouldn't be able to see online.

I stopped by the other forum and didn't see any of your topics. If you have any links to them, feel free to PM me here and I'll drop a note for you.

Well, we ran the December Maint release last night, and that had no effect. I think it may be the March release that we really need, but we wanted to do them in order so the March release will be done next week.

I did however manage to blow up my callflows in VMPro, I had saved the configuration but not exported the callflows (inexperience showing). Is there any way to get them back without rebuilding it?

Thanks, Dawn

ipofficeguy, thanks. I had my boss set up another user for me from her home, but I am hesititaing to use it because they obviously have all my IPs and domains. The thread I am most concerned with is https://www.tek-tips.com/viewthread.cfm?qid=1457999&page=2

I think we really need the 4.1.12, so next week I'll be doing that one.

Ok, found a backup DB for my VM call flows, thank goodness!

I have now established a better workaround for our Hwy location. I have set up 2 extensions that forward to the full 7-digit numbers for the spokes where the calls drop. The eliminates the need to transfer to the Hub first.

Notably, the forward number has to be a full 7-digits to prevent drops. If I use only a 3-digit extension, the calls will drop.

Hey Dawn,
If you are going to do an upgrade what I suggest is downgrade it first all the way back to the release it came with from the factory. Then re-upgrade it following all the proper documentation on how to accomplish this. With your BP, I would not assume that it was done properly regardless of if there is a CQ for your issues or not. It will take a little longer, but well worth the time to know it is all done correctly, so you have no firmware issues.
The IPO will allow you to skip firmware releases that are required for proper upgrading via the upgrade wizard. The only real upgrade wizard is the one clicking upgrade on the IPO. With your BP, I would not assume anything which can be done wrong was done in any other way without checking it. In the case of a firmware upgrade being done properly the only way to know it is done correctly is to do it yourself.

aarenot, I have already done the upgrades, they didn't help. I would be willing to try what you suggest though, but how would I find out what release it came with? I have no idea what I'm doing on that level.

I would guess it is 2.1 originaly from how long it has been in. You might ask your BP tech he may recall. The downgrade process is then same as upgrade, pnly load the version of manager one release back to downgrade the system. I forget what release you are at, but the key releases to be sure are done are the intermediate releases(999), and any releases that are required prior to upgrading to the intermediate releases. Some releases had to go to a certain maint release before they could be upgraded.

On second thought, do you know what a DTE cable is? It might be easier to just wipe it clean, and reload the firmware that way. Do be careful with doing this, and read up well before trying.

We had the Same issue. We found that on the Firewalls that we had VPn'd that it was inspecting H225/245 traffic and we needed to disable the inspection on both firewall ends. we also had to make sure we are on software version 4.2 latest patch.

just thought i would post our resolution here since we didnt really see one.