RS232 Troubleshooting

[gar]

070718-1146 EST USA

A little over a year ago we sold several of our I232 systems to a customer.

This customer has three HAAS machines. One is a VF2 with high speed machining, but a standard spindle. The VF2 is used to machine a part from a solid block of aluminum. Total time including a number of separate setups is currently about 8 hours.

When we sold them our isolator systems they had very great difficulty in sending programs to the VF2 because these are large programs requiring DNC (drip feed). Great difficulty means the program would crash many times during a run.

The VF2 is sufficiently new as to be using vector drives. We know these vector drive machines generate a substantial amount of electrical noise.

After installing our I232 Isolators their problem was solved, and I understood that they were able to operate at 115.2 kbaud. Today it is not clear that they did operate at 115.2 kbaud, but I believe it was the case. All communication was by phone.

However, a short time after the year ago installation I was called and told that transfer failed about half way thru their program. The failure point was quite consistent, probably precisely at exactly the same program point. They sent a copy of the program and I could reproduce the problem on our machines. The problem was related to program line numbers. If you exceed a line number somewhere around 100,000, then HAAS crashes. Removing line numbers from their programs eliminated the problem. Somewhere else I believe I described this. The precise number of line numbers where this failure occurs is not clear in my mind.

At the year ago time and up until yesterday my only contact with the customer was via telephone.

About 2 months ago I received a call saying that they had communication problems. I was unfamiliar with the communication program they were using, so I suggested they try a 30 day free trial of Cimco Edit. There was no indication that our Isolators were the problem, but one never knows. However, we do have a general knowledge of the reliability of our system.

They tried Cimco Edit and it either did not work at all or did not solve the problem. I have experience of extremely high reliability with Cimco Edit for communication at 115.2 kbaud. We use it for final testing of our Isolators at 115.2 kbaud.

About this time the customer had HAAS service come in. Any information about what happened is foggy. Somewhat after this I got a call telling me that the maximum operational baud rate was 9600 without error. But above that speed there were data errors. This did not correlate with our knowledge of what to expect with the I232 System.

They were not interested in paying for a service call. I told them I would stop in sometime when I was in the area. yesterday I stopped by.

So the question is does the I232 System have some problem in itself (virtually impossible based on the error free operation at 9600), or in the interface between either the VF2 or the computer?

I know what the interface voltage levels are at the VF2, but I did not know at their computer. A check at the computer showed -9 V at rest, and 0 V with data transferring. Thus, no problem.

Their setting were:
At the computer 7 data bits, 1 stop bit, even parity, XON/XOFF, 9600 baud.
At HAAS 7 data bits, 2 stop bits, even parity, XON/XOFF, 9600 baud. Note the disparity in stop bits. Later I proved this was not the problem.

The HAAS memory size is 1 megabyte. I changed the HAAS stop bits to 1. Then, ran a send test to HAAS memory (not drip mode) at 115.2 kbaud. This loaded error free. The file size was 1.4 megabytes, and used 95% of HAAS memory after loading. Note there is compression on loading a file. Probably because values are stored as binary in HAAS memory vs ASCII decimal in the RS232 data stream, however, this is a function of internal word size. And the reduction of commands to a simpler storage. For example G codes might be stored in 8 bits internally, but use 32 bits in the input stream (three 8 bit characters plus space).

Reloaded a copy of Cimco Edit and it provided successful transfer at 115.2 kbaud.

Next I changed HAAS back to 2 stop bits. No problem. This means in receive mode the UART in HAAS does not require 2 stops and will work with only 1 stop bit in received even though you have programmed HAAS for 2 stop bits.

Note: the above tests were a direct loading to HAAS memory and not during drip feeding. No indication of the previously described error problem above 9600 baud. The errors they had were corruption of data in HAAS and no errors in the source file in the computer.

To save time because they needed to make parts I disabled FIFO at at the computer. This is done in Device Manager. This is because it could be a problem in drip mode, but i have not tested for this.

Next I had them use Cimco at 115.2 kbaud and drip feed to the machine and make a part. I went to a long lunch. An hour and a half later I went back to the plant and the machine was still running, by now all the the program had transferred into HAAS memory and there were no transfer errors. Note: once transfer is complete you can use the computer again. You might be able to before transfer is done, but this will probably slow down data transfer because of timesharing.

More information:

Anytime they were having problems with corrupted data there was never a time when there was a parity error. This means there were no errors in the communication channel including the UARTS at both ends. The problems were inside either the computer or the CNC excluding the UARTs. There has to be a parity error if data is corrupted between the sending UART and the receiving UART. There are rare cases where this is not correct, but extremely rare. In both HAAS and the computer the parity is generated in and checked in the respective UARTs. In an Okuma I believe the parity check is done after the UART.

When HAAS came in about 2 months ago they reloaded something in the HAAS machine. Whether, high baud rate and/or Cimco was tried after this is unknown. Was the problem causing data errors in the HAAS I do not know, but I suspect this was the error location.

Why is using a parity check useful? Suppose I send the ASCII character numeral 2 which is 32h or 00110010b and there is a 1 bit error such that the binary value becomes 00110110b or 36h or 6 decimal, and the number is in the inch position, then the commanded move is 6 inches instead of 2. With parity check enabled this will be detected and the program can not be executed.

.

[gar]

070724-0820 EST USA

Follow-up:

I called the customer yesterday.

They have been running continuously now for about 1 week at 115.2 kbaud with zero errors.

In retrospect I believe that after reloading HAAS or whatever else they may have done several months ago that they never tried Cimco again and/or running at 115.2 kbaud. Also we do not know if my changing the computer UART programming to "disable FIFO" had any effect.

My guess remains that within HAAS there was some problem corrected by the reload.

.

[gar]

070731-2010 EST USA

New information:

Today I got a call from our customer again. Since I was there about 2 weeks ago they have been drip feeding at 115.2 kbaud 24 hours per day with no errors.

This weekend their trial version of Cimco Edit timed out. They called Cimco and purchased a copy. This means they were given a new key to permanently enable the software. Loaded the key and started to run. Problems like their earlier situation. Note: nothing was done to the Cimco software except load the key. On checking all communication parameters were unchanged.

Two weeks ago when I visited their shop the garage door was open and ambient temperature was comfortable. Today we are near 90 deg F, but they are running at 70 deg F air conditioned.

There are still no parity errors. Thus, almost a certainty that the problem is not in the UARTS, TTL to RS232, or our I232 link. If there were low density random errors, like 1 in a thousand, in the communication channel, then virtually all should be flagged with a parity error.

The type errors that occur are typically format flagged by HAAS. Extra numbers after the decimal, and other changes. Inspection of the source code on the computer shows no errors in the source.

My current contact is via phone so information is second hand.

Remember their HAAS memory is 1 meg, and they have no programs stored in the memory. Thus, the entire program memory space is available for the DNC buffer.

They are back to running at a low baud rate to get error free transmission. I had them change back to 115.2 kbaud, and find a file of about 1 meg in size. I had them load this in program memory. In other words non-DNC. There were no errors. Then tried to load the same program in DNC mode. Errors right away. We are estimating at about 128 bytes. Back to program load mode and there were no errors. Next we found that up to 19.2 kbaud DNC was error free. At 38.4 errors again. Remember for the last two weeks DNC was error free at 115.2 kbaud.

My guess the problem is again HAAS, which would mean the original problem is still present. I know that I can send large files to HAAS at 115.2 kbaud with virtually no handshake delays. This is file transfer. I do not know if that is the case in DNC when one is filling the buffer and not running the machine, our condition where the errors occur. I think it is only coincidental that failures started after loading the new Cimco key, and therefore unrelated.

Why do they call me first instead of HAAS? Probably because I help them analyze the problem. They are going to call HAAS. I hope HAAS brings a hair dryer and some circuit cooler because this may very well be a thermal problem. There could be power supply problems.

At this point it seems like HAAS has different code for the program load mode vs DNC mode even though these are nearly the same function to get incoming data in memory.

.

[serviceman]

i always recommend that people use xmodem especially on long drip feeds xmodem is available in hyperterminal which is standard with any windows operating system. x modem has error checking and will retry sending packets if they fail up to like 3 retrys per packet not sure. the good thing about xmodem is that you only need 3 wires and a shield. error checking is always better than sending bad data to the machine. with xonxoff the data is sent whether the machine received it or not and never verifies the data sent.

[serviceman]

always try toggling dpr serial in parameter 209 or 278 ? if it is available. this reroutes the serial data. the later machines did away with it i think after 13 series software.

[gar]

070801-1920 EST USA

serviceman:

XMODEM is better than XON/XOFF with parity check since it has retry capability. Otherwise both only require 3 wires for a direct connection.

If XMODEM as implementated in HAAS included a check of the data after it was stored in its final buffer location that would be good. I doubt it does. I suspect that the XMODEM check only extends into the first buffer loaded.

In the current problem I am describing there are no errors anywhere in the path including the computer UART all the way thru the HAAS UART because there are no parity errors indicated.

My guess is that there are problems within the HAAS CMOS memory or other locations, the processor, and associated circuits. Depending upon where the problems are XMODEM might not catch the errors.

However, it may be worth trying it.

.

[serviceman]

i have not seen it but i have heard from some people that they needed the other wires i am not an expert on rs 232 from memory i think pins 5 8 and 20? not sure dsr and dtr cts i would have to look it up but i have run xon/xoff with just the 3 wires. but the error checking does go on throughout the send cycle that i am sure of. i have seen failures do to bad connections or grounding very late in the program and you can see the retries and resent packets in hyperterminal and this could give you an indication of what is happening good luck.

[gar]

070801-2000 EST USA

serviceman:

Pins 6, 8, and 20 are internally jumpered in HAAS and go nowhere. When we make a connector we jumper 6, 8, and 20 and separately 4 and 5 if it is for software handshake which includes XMODEM. On a HAAS there is no need to jumper any of these pins for software handshake. We do this for compatibility of the cable with other machines such as Fanuc.

On the current problem that I am describing if XMODEM did solve the problem as a result of its use it would be the wrong solution. I believe at this point in time that in this HAAS machine there is a hardware failure that may be presenting itself as a software failure. This needs to be determined because it could be a precursor of other failures.

When I was discussing were the error checking is done with XMODEM I was referring to at what buffer stage in HAAS the checking was done.

In XMODEM a packet is sent to the destination. Here a buffer of adequate size is loaded with the packet. After a packet has been fully received it is checked for errors. In this packet in this buffer is exactly the same sequence of bytes in their original form from the sender. If no errors, than an acknowledgement is set back to the sender indicating it is ok to send the next packet. Otherwise the sender is told to resend the packet.

In the HAAS program memory space I am fairly sure that some bytes are remove from the source, others are converted to an internal HAAS format. This of course would be after the XMODEM buffer. For example there is no reason to store DPRNT as 5 bytes internally in HAAS. This takes too much space. A number like 25.4356 is probably converted to a 32 bit number in 4 bytes instead of 7 bytes of ASCII code. So effectively compression is preformed. This may not be exactly what they do, but you can demonstarate that there is compression as follows. Empty all programs from a machine with 1 meg of memory, then load a typical file of maybe 1.4 megs of data. It will load and not quite fill the 1 meg.

There might be some combinations where this is not true. If you loaded a lot of small increments like x5 x7, then each of those single ASCII numbers which take 1 byte (8 bits) might actually require 32 bits in HAAS.

Back to XMODEM. The data in the XMODEM buffer which has been checked as good is now probably compressed to HAAS internal format for storage in the executing buffer. If there are memory error problems in this area they will not be detected by the XMODEM function. However, if this data was re-expanded it could be compared with what is in the XMODEM buffer. Highly unlikely this done.

.

[giantcake]

Gar,

You mentioned earlier in your posts that you disabled the FIFO buffers in the PC WinXX OS software. Perhaps when loading the CIMCO serial number the program then goes and sets internal parameters back to default or the program goes out and checks for WinXX defaults and sets them.

Since nothing else changed except for the data rate at which they can operate, I would suggest a look at the software and/or settings on the PC one more time.

Also, I note your remarks about compression. Keep in mind that computers do not store the start, stop, or parity bits from the serial data stream. Just the 7 or 8 data bits. The start, stop, and parity are typically generated by the UART chips. Thus you would basically achieve 30% compression of the transmitted data when stored in memory. (i.e. 1-Start, 1-Stop, 1-Parity bit stripped, 7 data bits remaining).

Although I am not a Haas expert, my thought would be that Haas would have updated the firmware when they came out if this was a line numbering bug and startup diags would have detected most faulty memory. (I hope anyway)

I'm curious to find out what the solution is as I do work on serial (and network) comm protocols for a living. CNC is a hobby for me.

-Steve

[joecnc1234]

can the machine move faster than 9600? if not why go 115.2k? In my experience the faster baud rate only leads to buffer overload if your having a pissing contest with haas good luck.
Joe

[gar]

070802-0612 EST USA

giantcake:

FIFO disable is a Microsoft parameter set under Device Manager. This is not controlled by Cimco software. However, the other parameters baud rate, stop bits, data bits, and parity are setup from Cimco.

Note: Currently direct loading of a program to HAAS program memory produces no errors at 115.2 kbaud, whereas in DNC mode errors occur at 38.4, but not at 19.2. For the past two weeks prior to this last weekend there were no errors at 115.2 in DNC mode running 24/7.

The compression I am referring to has nothing to do with what occurs between and including the sending UART thru the receiving UART. The comparison reference is between the file size in the computer and the amount of memory used in HAAS program memory space.

The line numbering bug is not the cause because that error does not occur until about 100,000. The current errors occur early in sending a program, like a few hundred bytes, and there is no essential error difference between sending the program with line numbers and without.


joecnc1234:

Yes the machine can be starved for data at 9600 baud. Most modren machines are starved at this rate in mold making or other contouring applications in portions of the program. One of our customers improved their net thruput in making carbon electrodes by a factor of about 2 after we installed our I232 system that allowed them to operate error free at 38.4, the max for their CNC. Prior to this they had machine stutter problems at 9600 and also errors. When you save machine time like this it translates to major cost savings, and getting the job done quicker.

Faster baud rates do not cause buffer overload if handshaking is working correctly. The purpose of handshaking is to prevent buffer overflow.

.