IBM 5251 terminal USB conversion (but not what you think)

[inmbolmie]

This is a continuation of this post where I describe an IBM 5251 terminal I recently found. It is about a USB conversion, but not what you think...

terminal.jpg (241.32 KiB) Viewed 47455 times

This terminal is in good shape and doesn’t need a restoration at all, it doesn’t even need a deep cleaning, so an obvious question was what was I going to make with it to have some fun.

One possibility was the usual direct USB keyboard conversion (commonsense, xwhatsit) to use it in a modern machine, but I already have a converted Beamspring and I would have no use for another one. I also didn’t want to throw the keyboardless terminal away once the keyboard was converted.

Another possibility was plugging the terminal itself to something to use it, but I would need a System 36/38 or an appropriate AS/400, that I don’t have space for or desire to get. There is no way (or at least I haven’t discovered it) to plug this thing to a modern system, for several reasons:

Twinax_Stecker.jpg (223.49 KiB) Viewed 47455 times

• The physical interface is Twinax cable, that is a weird dual core shielded cable similar to coaxial that is nowadays very obsolete. There are twinax ISA cards out there (I have one for an IBM 5160), but there is another problem with them.
• The second problem is that the 5250 protocol is master-slave and the host acts always as a master, so even if you plug two terminals together or a terminal and a PC with an ISA card there is no way the terminals and the PC can talk to each other. The PC software available is terminal emulation software, not host emulation. The only hosts available are “real” hosts and as I said I don’t have one.
• The third problem is that the 5250 terminals use EBCDIC character encoding, essentially proprietary IBM shit that was very different from the standard ASCII.

So I felt I only had a very heavy dead weight catching dust, beautiful but useless. I wanted to change this so I started this project to make a custom conversion with this objectives:

• Build a physical interface to convert from Twinax to USB.
• Build a protocol converter to read and write 5250 frames from/to a PC.
• Program a terminal converter to adapt a Linux-friendly terminal to the available 5250 commands and EBCDIC charset.
• Post an entry in desthority,net from the terminal (!).
• Entertain me during the pandemic.
• Do not burn too much money in the process.

I will leave the technical details for a later update, but I have managed to get all three components working, here is the converter prototype:

conv1.jpg (530.3 KiB) Viewed 47455 times

It is somehow like a “Frankenstein” of different parts: A physical interface based on a couple of SN7511X line drivers and a LM361 analog comparator, the wire protocol converter is implemented using a Teensy 4 (very powerful and easy to program indeed, I’ve fallen in love with it), and the terminal conversion is implemented with a Python script in the PC side (for Linux only).

And this is the final result, a bash shell on the 5251 that is now fully, or at least 99% VT52 compatible.

cmdline.jpg (207.31 KiB) Viewed 47455 times

Positive aspects from the project:

• The 5251 is back to life.
• Best keyboard ever in its original package.
• Loud as hell.
• No need to modify the terminal, just plug to the standard Twinax connector.
• Can be used 152 meters away from the computer, if you have enough money to waste in such a cable.
• Learnt a lot of stuff in the process.

Negative aspects :

• It is still a heavy beast, I hardly can put it over the table.
• The converter needs external power.
• It is not the fastest terminal ever, specially for scrolling text, but it is perfectly usable
• Lacks some important characters on screen. For example, in my spanish version there are no dollar ($), pad (#), or exclamation (!) characters (c’mon IBM…).
• The original 5251 keyboard controller only supports three break keys (as those who use an external converter know well) so you have just one SHIFT, one CTRL and one ALT, and they have to end up in weird positions, so it’s not ideal for daily usage.
• Probably the most expensive and troublesome way to get a simple Linux bash shell nowadays.

Now some pictures and videos of the working terminal.

Closeup of the adapter.

conv2.jpg (502.5 KiB) Viewed 47455 times

The adapter wired to the computer, to the terminal and to an external power supply.

conv3.jpg (474.66 KiB) Viewed 47455 times

A couple of screenshots.

banner.jpg (253.34 KiB) Viewed 47455 times

browser.jpg (256.5 KiB) Viewed 47455 times

Quick video showing the command line and a couple of apps, including browsing deskthority.net!

Note the insanity of the way IBM implemented the audible alert (BEL character) using the keyboard solenoid rattling like a machine gun. The first time I hear it I almost fell off the chair.

And this is the real killer app for this terminal, cmatrix!

cmatrix.gif (3.65 MiB) Viewed 47455 times

As I said the technical details will follow in later updates, but I suppose that people that could be interested in replicating this can be counted off the fingers of one hand…

[AJM]

Wow! You are my hero!
I had exactly the same thoughts and wishes for such a project. The only difference is, that I don't have the knowledge and skills to make it a reality.
Should you ever consider building such a device for others, I would be very interested.
(BTW: I've heard of EBCDIC, because it was also used by Siemens for their BS2000 system. In my last job we had a customer, who used our software on such a system right up until 2018.)

[webwit]

Great project! And good to see DT is still Lynx compatible Figlet makes me walk down memory lane... made some fonts for that. I see myself telnetting into a MUD with that terminal and kill some orcs.

[inmbolmie]

Should you ever consider building such a device for others, I would be very interested.

Sure, I still have to make a PCB design, but I will get in touch when I'm ready to make the real board.

I see myself telnetting into a MUD with that terminal and kill some orcs.

That's a great idea that I will try for sure, some old-fashioned online gaming! And it gives me another idea I will try later...

[OldIsNew]

Awesome work - just too cool!

[Heikkonen]

This is some dope work you have done here! Im planning to restore 3741 to original working state,will take some time/effort along with reading tons of manuals.

[stoddartc1]

This is ridiculously awesome and something I didn't know I needed in my life

[JBert]

Oh my...

How did tagging work again so that we don't forget this in the next deskthority Awards?

[unoquepasa]

A very interesting project! Looking forward to those technical details, just curious about your approach to building that converter.

[SneakyRobb]

This is next level cool. Good job and very impressive!

[shine]

impressive!

[inmbolmie]

Thanks for your comments, are much appreciated.

For the technical details I will proceed in this way: First in this post I will comment the project in general as well as some design decisions, and later I will give the complete details about the software components as well as the physical circuit (But I still have to design a PCB)

So first I want to mention the documentation I’ve mostly used. The good thing about the IBM stuff of this era is that everything is well documented. For the 5251 this didn’t went to the extent of publishing the circuit schematics or microcode like for other systems (I’m still looking for them) but everything else is there, including complete specifications about the twinax protocol, frame format, 5250 command set, screen buffer and internal registers. The main source of all this information are the manuals published in

http://www.bitsavers.org/pdf/ibm/5250_5251/

bitsavers.png (33.4 KiB) Viewed 47055 times

But that unfortunately doesn’t cover all the details needed to replicate the hardware, as I imagine the main focus of that documentation was facilitate debugging and problem solving in the field. But I found another invaluable source of information in these old datasheets from the DP8344 communications processor.

http://www.bitsavers.org/components/nat ... 44BVJG.pdf

http://www.bitsavers.org/components/nat ... N-0641.pdf

bitsavers2.png (115.04 KiB) Viewed 47055 times

That was an old communications processor from the early 90’s designed precisely for this task, making interfaces to IBM 3270, 3299 and 5250 systems. Those documents give more low level information and implementation details, as well as crucially a physical interface design I used as a basis for the project.

After “processing” all this information I got a clearer picture of how to proceed.

• Use the interface circuit design from the DP8344 datasheets to connect the converter to the Twinax signal. The other options were: One, reverse engineer the IBM circuit, but lacking a proper schematic this was going to be painful. And two, design a circuit from scratch, but that was out of question as I currently don’t have an oscilloscope (as it died some time ago), so I wanted to try first with a proven design that doesn’t need to be debugged at the signal level .
• Use “something” to interface the digital NRZI signal (I will give later details about that) that comes/goes from/to the physical interface and convert it to USB
• Use a host that can provide a command line and implement the necessary conversion from a standard UNIX terminal to the 5250 protocol

The first idea was joining the second and third components in one, and use a Raspberry Pi, as it has the capabilities to provide the shell and terminal protocol conversions, as well as a GPIO interface to connect it at low level to the interface circuit. But after some testing I decided not to follow this route. The reasons can be summarized like this:

• The GPIO interface accessed from a user space program is slow as molasses and totally unsuitable for this task. The NRZI signal is 2Mhz and you can get nowhere near sampling at that speed with a user space program.
• Even if you have external circuitry to sample and try to transfer it “in parallel” the GPIO has no facilities for that and I doubt that can be used efficiently for this.
• For it to have the minimal chance to succeed, the program has to be implemented as a kernel module in Linux, and doing that it would be nearly impossible to port it to another system or make the project minimally fun.

So with the raspberry discarded, the next possibility was using a FPGA device. Those are inexpensive (at least at the level needed for this project) and have all the needed power as the “program” is implemented in hardware. For sure the interface can be implemented using a FPGA but I had no prior experience and those things are fairly intimidating, so I reserved this as a last recourse if everything else failed. This would be the “very professional” approach.

The last option was using any of the popular microcontroller platforms (Arduino, Teensy, etc) that are cheap and come in a variety of form factors and capabilities. Immediately the Teensy 4.0 caught my attention:

• Already has a high-speed USB interface.
• Small form factor.
• Very easy and friendly to program and debug, and well supported by the community.
• Has TONS of power as has a 600MHz core (!) and that was a key factor in the final implementation.
• Lots of connectivity including a lot of high speed digital ports, I2C, SPI, analog converters etc.
• Cheap (~20-30€) and easy to get from a variety of vendors.

Teensy4.png (448.73 KiB) Viewed 47055 times

Just the last possibility was using the vintage DP8344 microcontroller itself, that still can be obtained from some ebay sellers. But I didn’t really consider using it as I would have to learn its assembly language and that would be wasted time, as that knowledge wouldn’t be of any use out of this project.

For the last component, the protocol converter from/to 5250, I wanted to use a simple approach. Technically this could be as well included in the Teensy, but you will need to connect it anyway to something that can provide a shell, so there is no reason not to move this part to the final system, where it would be far easier to program. Due to its popularity and the portability it provides, I thought that a Python script would do the job here.

So after all of this, we have the basic design of the converter:

• Twinax to digital NRZI interface circuit from the DP8344 datasheet design.
• A Teensy 4.0 to read an write the NRZI signal and send it back via USB to a Linux machine.
• A python script to manage the USB communications, spawn a shell to show in the 5251 and convert from the 5250 protocol to a UNIX terminal (and from EBCDIC to ASCII) and viceversa.

The next part will be the detailed description of the physical interface and signals involved.

[pcaro]

What a awesome project!

[inmbolmie]

Now the details for the “layer 1” of the converter, the physical interface to the Twinax terminal.

For the explanation about the Twinax signal and cabling I think that the general description from Wikipedia is fine and I wouldn't explain it better, so I copy the relevant parts here from https://en.wikipedia.org/wiki/Twinaxial_cabling:

The data transmission is half-duplex, balanced transmission, at 1 Mbit/s, on a single shielded, 110 Ω twisted pair. [...] Twinax is a bus topology that requires termination to function properly. Most Twinax T-connectors have an automatic termination feature. [...]

Signals are sent differentially over the wires at 1 Mbit/s (1 μs/bit ± 2%), Manchester coded, with preemphasis.[...] to provide preemphasis, for the first 250 ns (1/4 bit time) after a signal is driven low, the negative signal line is driven to −1.6 V. [...]
The two wires are denoted A and B. To encode a 0 bit, A>B for the first half of the bit time, and A<B for the second half. A 1 bit is the opposite. Thus, each signal line is driven low for either 500 or 1000 ns at a time, of which the first 250 ns is emphasized.
This is a picture of the teoric waveforms from the DP8344 datasheet.

twinax_waveforms.png (64.84 KiB) Viewed 46913 times

The important part here is understanding the “Manchester” signal (it is named NRZI in the DP8344 datasheet but that’s incorrect). In such a coding, the logic “one” is encoded as a transition from one to zero in the middle of the bit, and the “zero” is encoded the opposite, with a transition from zero to one in the middle of the bit. At least this is the convention used for our system for the transmission “over the wire” (sometimes can be confusing identifying if we are speaking about the inverted or not inverted signal).

This guarantees at least one transition for every transmitted bit, or said another way, that you won’t see the signal at the same level for more than two half periods of transmission. This is important because this allows to recover the clock from the signal and this way synchronize the receiver and the transmitter.

Although the binary symbol rate, the speed at which we send information, is 1Mhz, the Manchester signal is really 2Mhz due to the half bit transitions. That is why we will talk about “half bits” where for a logical “one” you will have to transmit the two half bits “one + zero” and for a logical “zero” you will have to transmit the two half bits “zero + one”.

So for example in this pictured signal…

manchester.png (3.73 KiB) Viewed 46913 times

This is the inverted Manchester signal, so you can see five “ones” at the beginning of the transmission, that is, five transitions from “zero” to “one” in the signal (as it is inverted the sequence is the opposite of the non inverted signal).

The objective of the interface circuit is:

• For the reception branch, recover the 5251 transmitted Manchester input signal sent differentially over the wires A and B. This can be done subtracting the analog signals, B minus A
• For the transmission branch, generate the signals A and B from our Manchester output signal. For this you would also need the Manchester signal itself (inverted), plus a copy of the Manchester signal (non-inverted) delayed 250 nanoseconds (a quarter of a bit) to generate the preemphasis at the beginning of each transmitted bit. The preemphasis is a technique to improve the signal transmission boosting some parts of the signal.

So the Teensy ends up doing a simple task, send and receive those Manchester signals, that are digital and so easy to generate and read, and the circuit will do all required conversion to put them on the wire.

As I said I for this purpose I used the reference design from the DP8433 datasheet.

dp8344_circuit.png (99.33 KiB) Viewed 46913 times

The main parts are

• One SN75110A and one SN75112 drivers to generate the Twinax waveforms A and B
• One LM361 analog comparator to recover the Manchester signal substracting A from B and outputting the result with a 3V digital level
• A couple of 54,9 Ohm resistors for optional end-of-line termination.
• Some logic gates (AND, XOR, NOT) for glue logic

The signals in the circuit are:

• TX-DATA is the inverted Manchester output signal
• TX-DLY is the non-inverted Manchester output delayed one quarter of a bit (250 nanoseconds)
• We put TX-ACT high to enable transmission and low to disable it.
• We won’t be using the PWD-GOOD signal
• NRZI-DAT (we’ll call it RX-DAT-INV instead) is the inverted input decoded Manchester signal
• Phase-A, Phase-B and SHIELD-GND go to the Twinax connector

This is one real capture of the digital signals using a logic analyzer, it pictures a request frame transmitted to the 5251. In the capture we show TX-DATA both inverted and non-inverted:

transmission.png (5.61 KiB) Viewed 46913 times

• Signal 15 is TX-ACT (pin 4 of the Teensy)
• Signal 14 is TX-DATA non-inverted (pin 5 of the Teensy)
• Signal 13 is TX-DATA inverted (generated passing signal 14 through an inverter)
• Signal 12 is TX-DATA-DLY non-inverted delayed 250 nanoseconds (pin 6 of the Teensy)
• Signal 11 is RX-DAT-INV inverted (pin 7 of the Teensy)

Now the same capture but zooming out to see also a frame received from the 5251

reception.png (4.79 KiB) Viewed 46913 times

One drawback of this circuit is that you need four voltage levels, +12V and -12V for the comparator and +5V and -5V for the drivers and logic gates. Maybe with a different design that could be reduced, but now it is what it is.

This is my version of the circuit, (still work in progress):

5250_converter_sch.png (39.21 KiB) Viewed 46913 times

The final board when finished would be something like this:

5250_converter.png (145.76 KiB) Viewed 46913 times

About the modified circuit:

• I ignored the PWR-GOOD signal as I don’t feel it’s needed
• I use 74HCT series chips for the logic gates, to convert from CMOS (Teensy) to TTL level for the drivers. Just in case.
• The output of the LM361 needs no level conversion for the Teensy as it is already 3V compliant.
• I generate the +-12V signal using an isolated DC/converter (DCW03A-12), and from that the +-5 levels from linear regulators (LM7905 and LM7805). That converter is expensive but you can use as a power brick almost anything that outputs between 9V and 18V, and the isolation gives extra electrical security.
• The termination resistors are implemented using 100 Ohm potentiometers.
• The board provides two possible ways for the Twinax connection, one via a 3 pin header and other via a DB15 for an external T connector. For the latter you have to disable the internal terminators removing two jumpers, as T connectors are usually self-terminated.
• The Teensy is powered from the USB port
• We use one digital input (RX-DAT-INV) and three digital outputs (TX-DATA, TX-DATA-DLY and TX-ACT, all three non-inverted) from the Teensy

One detail that I found hard to find is the assignment of the A an B signals to the correct Twinax pin, as they are not reversible. I found no place in the documentation with an unambiguous assignment of signal to pins. So for future reference this is the correct polarity for the female connector as seen from the front of the terminal:

EDIT: the prior pictures were wrong, I'm updating this post to correct that using an "official" document from IBM I've found

phases_ok.png (119.94 KiB) Viewed 46490 times

In a real system this is irrelevant, because as long as you do not cross the conductors when assembling the cables you will always end with the correct polarity (the conductors are usually color-coded) but for me it was a small headache finding it.

Finally, I don’t have a Twinax cable, I ordered one on Ebay but due to the pandemic is is “trapped” in my office and only god knows when I will be able to get it. Because of this I provisionally connected the converter directly to the 3-pin header where the Twinax cable goes in the 5251 main board. It should work exactly the same with the real cable, as the circuit termination is inside the main board, and I’m using a moderately long cable.


EDIT: also correcting this picture

IMG_7552_OK.jpg (556.74 KiB) Viewed 46490 times

Next post, frame format and data link layer code for the Teensy.

[eeguru]

I'm very interested in this project. I've been working on the problem for some time in reverse. The intent is to create an open-source hardware bridge design from a twin-ax port on a real midrange system (my S/36 in this case) to either a USB terminal emulator or generic TN5250 bridge device. I'm implementing everything in an FPGA+softcore atm. I've captured several traces of my S/36 IPL'ing with a 3179 terminal attached. I've managed to only get so far with decoding the protocol. Mostly from the 'Maintenance Information Manual' pages 169-184 (PDF page #s) / 4-9 through 5-4 (IBM page #s).

I think that information might have been enough to get your project to where it is today. However it's incomplete. Most of the command codes' arguments are undocumented in that manual. I've looked through many other doc sources with no luck. Are there any docs/pages in the bitsavers archive you've linked above I should explore further?

I've uploaded my capture/analysis C code as well as the decoded IPL traces to:http://www.retrotronics.org/tmp/s36_ipl ... 1dec19.zip

Are you planning on releasing your Teensy source? I'd love to collaborate on project that would be a generically useful emulation of either the system or the terminal side of the twin-ax link (or even printer).

[eeguru]

Just to elaborate a bit more. There is a ton of information about the Model 11 synchronous serial protocol format in many manuals/sources. However the transport layer and basic command layer is vastly different to TwinAx. The serial/SDLC side of the terminal operation is an early implementation of IBM SNA - mostly LU 6. The TwinAx side is much simpler but less documented. Some things I presume are shared like field attributes. But it's difficult to determine the shared cross over points between SDLC/SNA and TwinAx.

To implement the terminal side to a genuine Midrange system requires implementing a persistent frame buffer with attributes and supporting field delineation and read-back by the host. Features I have not found a good reference for...

[inmbolmie]

Are there any docs/pages in the bitsavers archive you've linked above I should explore further?

First, I have to say that I'm not familiar with all those IBM protocols, outside the very narrow knowledge needed for this particular project, and a month ago not even that.

There is one document referenced by many others, "5250 Information Display to System/36 and System/38 System Units Product Attachment Information" that I've been still unable to find online, and maybe it is what you are looking for.

I've uploaded my capture/analysis C code as well as the decoded IPL traces to:http://www.retrotronics.org/tmp/s36_ipl ... 1dec19.zip

Yeah , that is exactly the same kind of dumps I'm making, everything looks familiar, except for the READ commands. I do not use them at all because I keep track of everything I need locally (cursor position, etc) and I don't need to read anything back from the screen buffer or registers.

Also it seems that that system has much bigger reception buffers, and the "WRITE CONTROL DATA" command has more functionality that I know because I only write 1 byte with that command.

Your task is more complicated that mine because as a host I decide which commands and functionality to use, while as a terminal you have to implement everything that the host throws at you, even undocumented stuff.

Are you planning on releasing your Teensy source? I'd love to collaborate on project that would be a generically useful emulation of either the system or the terminal side of the twin-ax link (or even printer).

Yes, but be aware that that code is very very simple indeed, and you seem to have mastered frame capture already. I'm debugging now some last minute glitches and I will publish that once they are solved.

[inmbolmie]

It’s time for layer 2 components, essentially frame format and Teensy code.

About the frame format, it is really well documented in the sources linked before, and there isn’t much that I can add here to that, but I will give a brief description.

Each frame is 16 bits long (or 32 half-bits).

Each transmission over the Twinax medium is composed of:

• A 22 half-bit fixed start, bit-sync and frame-sync sequence (“1010101010111000” for the non-inverted signal). Can’t be decoded to bits because it contains the “invalid” sequences 111 and 000 to differentiate it from regular data.
• The 16 bit or 32 half-bit frame itself, composed of:
  • 1 fixed sync bit (2 half-bits “10”)
  • 8 data or command bits (16 half-bits)
  • 3 station address bits (6 half-bits)
  • 1 even parity bit (2 half-bits)
  • 3 fixed end of frame 000 bits (6 half-bits “010101”)
• If the message is a multiframe message, the next frame is separated from the previous frame by a configurable number N of bits “0” (N times “01” half-bits). We will configure N always as zero (using the layer 3 SET_MODE command) so in our case the next frame will start immediately
• Up to 256 frames can be concatenated in this way
• The last frame will have the station address 111 as an end-of-transmission indicator

From the IBM Maintenance Information Manual

frame.png (119.78 KiB) Viewed 46678 times

So each frame carries 8 bits of data, that can be a command code, a parameter to that command, a response status byte, the content of a register or a received scancode depending on the command we are sending. So a multiframe command can transmit from 1 up to 256 bytes of information.

The protocol is master-slave, so the controller will start a transmission first sending a message for the station addressed in the “station address” field (values 0 to 6 are possible for a maximum of 7 connected terminals) and the addressed station the has to initiate a response in a maximum of 80 microseconds. If no response is received in 80 microseconds it is assumed that there is no response from the station and a new transmission can be initiated by the controller.

req_resp.png (17.71 KiB) Viewed 46678 times

Those are the signals we have to generate and process in the Teensy.

To implement this processing, specially for the reception branch, there are different possible approaches. Some will rely more in hardware to synchronize to the incoming signal, sample it and buffer the incoming frames, but I wanted to try a fully software-based solution as the Teensy provides plenty of processing power for that.

So the idea was sampling the incoming 2 Mhz signal directly from the Teensy. For that, the program really oversamples it at 8 Mhz, taking three or four samples per half-bit. As the sampling clock is not synchronized to the incoming signal, there will be a hugue clock drift, but we will always end having a minimum of 3 and a maximum of 4 samples per bit. As the Manchester signal has guaranteed transitions each bit, the program will be able to constantly resync sampling considering that 3 or four consecutive samples are part of the same half bit.

The Teensy 4 is so powerful that, with a 8 Mhz sampling rate, (125 nanoseconds cycle) between two samples we have still enough cpu cycles to do all the inter-sample processing, that is start sequence detection, bit decoding and frame assembly and even some integrity calculations. This approach won’t be possible with other slower systems.

This is a very simple approach indeed but in practice it works very well.

There is another feature that facilitates this implementation, that as a master-slave protocol we don’t need to be constantly waiting for incoming transmissions and collisions are not possible in this protocol.

Given this, the code is very simple and it is structured around an infinite loop with four sequential steps:

1. Listen the USB port for incoming commands from the host
2. Create and transmit the frames for the incoming commands
3. Wait if necessary 80 microseconds for a response, and if during that time we detect the message start sequence, sample the frames and assemble the incoming message data.
4. Transmit, if received, the incoming message data back through the USB port

To help during development I’ve implemented a “delay in sampling” detection alert, that will light up the Teensy LED if the program detects that too many cycles have passed between taking two consecutive samples. The processing code had to be optimized until the LED never lights on while on operation.

I have uploaded the Arduino source code to this repository:

https://github.com/inmbolmie/5250_usb_converter

Note that this code will need the Teensy to be configured to operate at 600Mhz to work.

To program it you simply need a usb connection to the Teensy and the Arduino IDE with the Teensyduino addon, so it is very easy to work with.

I know that this approach is not 100% optimal, because it would be very convenient to offload some “layer 3” stuff like polling management to the Teensy. But that in some way will break the current clean layer separation (layer 1 for the circuit, layer 2 for the Teensy and layer 3 for the Python script).

Another feature is that the communication “protocol” between host and Teensy is text-encoded. This allowed me to test the Teensy implementation from the Arduino serial console issuing commands by hand well before I had the Python program to control it. I left it his way in the final system as it is not a performance-critical component although the encoding itself is slightly weird.

Sending commands to the 5251 and getting responses from the Arduino serial console

teensyduino.png (128.73 KiB) Viewed 46678 times

Next part will describe the Python script for terminal protocol and character set conversion.

[eeguru]

Thanks for sharing. I don't have a lot of time to work on my related project right now, but I've been thinking about it a lot.

One of the biggest issues I saw when passively sniffing the line between a real S/36 and different terminals is lack of hysteresis in the receiving circuit. Like you, I modeled both side of my phy circuit from the app note you quoted. The differential receiver does it's job well. Depending on which A/B side is higher in voltage, the open-collector clamp comes down hard or not.

The trouble I found is the idle state of the line varies depending on which side is active. It's not a true NZ signal - and was never meant to provide a recoverable clock to the down-stream. When the data stream is idle, there are no line transitions - not even a constant stream of payload 1's or 0's - which would produce an alternating line level suitable for a downstream PLL/DLL. The diff pair is near ground level with some variance - usually tens of millivolts difference (if that) - before the sync pattern. When the S/36 drove the poll command, it's idle levels were oriented so the diff receiver picked up a small difference in one orientation. Then when it completely high-z'd the pair to let the terminal potentially drive a response, the line floated to a different small offset. Then when the terminal activated it's transmitter but had not started driving the sync pattern, the diff pair was at yet another small offset. Often these three periods were at different polarities Only when the sync pattern started did the open-drain output have a definitive pattern - as intended. But the diff receiver has problems when the pair is idling at ground. It picks up slight bias and registers an analog change of just a few millivolts as a binary transition.

It's not a huge problem as the sync pattern should be the threshold of interpretation. But it would simplify logic/code if the signal had some sort of hysteresis threshold.

I thought about either converting the diff voltage offset to a ground referenced absolute voltage or current signal using an op-amp. Then adding a few analog comparitors to detect a bit, the sign of the bit, and the absence of a bit (idle). I've also thought about since I intend to do a Verilog framing layer, adding just one analog comparitor to catch that first emphasized pulse on the positive line as a data-valid strobe that gets renewed with every emphasized pulse until it times-out - delaying the data chain by a set time offset (with clock and FFs) to realign the strobe and data.

[inmbolmie]

... the diff receiver has problems when the pair is idling at ground. It picks up slight bias and registers an analog change of just a few millivolts as a binary transition.

It's not a huge problem as the sync pattern should be the threshold of interpretation. But it would simplify logic/code if the signal had some sort of hysteresis threshold.

Well, the circuit as I see it should provide a few millivolts of hysteresis, maybe you have to play with the values of the feedback resistors and lower it al little in your case. I would love to see the IBM reception circuit shematics anyway to know what their solution is. I didn't go for a PLL based solution trying to avoid precisely the kind of problems you are describing, but who knows, maybe with longer cables I will encounter also some problems. My plan here in case of transient problems was simply to remove some bits from the start sequence detection patern.

...adding just one analog comparitor to catch that first emphasized pulse on the positive line as a data-valid strobe that gets renewed with every emphasized pulse until it times-out - delaying the data chain by a set time offset (with clock and FFs) to realign the strobe and data.

I can't provide much feedback as I cannot really see the signals themselves, but I thing that trying to directly detect the emphasized pulse will only work for very short cables, with long cables the high frequency pulse will spread and be less noticeable

[DMA]

One of the biggest issues I saw when passively sniffing the line between a real S/36 and different terminals is lack of hysteresis in the receiving circuit.

I may have read something wrong, but it looks like you have 22 1.6V pulses - 11 on phase A, 11 on phase B - for clock recovery before actual transmission starts. How about just tripping at 0.8V and using that?

This is not a true differential signal either - lines are referenced to the shield. Looking at actual transitions - that 0.3V doesn't seem Terribly Important.
Just flip on the falling edge - like, same 0.8V - it will always be followed by "slightly low" signal. Talking back the same way could be a bit problematic - but it look like transmitter isn't too subtle. It just puts -5V into one of the phases, pinning the other to the ground - 12mA for the whole transmission duration, plus 27mA just for the first part of the pulse.

As for those millivolts when idle - if you're the last device on the line you should have those 55R pulldowns connecting phases to the shield, which would drain the lines of any residual charge pretty quickly.
And if you aren't - the device at the end of the chain should have it's pulldowns engaged, achieving the same effect.

[inmbolmie]

This weekend I've been trying the converter using an ISA 5250 emulation adapter card for IBM PC that I got some time ago. The card is hooked up to an IBM 5160 PC (with its proper model F XT keyboard to get the full retro experience) and a couple of interesting things came along with those tests.

First, I realized that somehow I screwed up the Phase-A and Phase-B ping assignments from the previous posts, so I've corrected that post and its pictures. The 5250 ISA card has a 15 pin connector whose pin assignments are well documented, so it became evident that I got it wrong when I tried to connect the converter to that card.

Second, I got it working with the new system and I made some adjustments to the Python script to support multiple terminals simultaneously, and then I got it working with the 5160 and the 5251 simultaneously. So I'm pretty confident it will work given the opportunity with the full 7 supported terminals simultaneously connected.

5160_AND_5251.png (2.1 MiB) Viewed 46492 times

Also the PC software for that card is supposedly emulating a 3196 terminal, so I imagine the converter will be also compatible out of the box with that system, but only using the features available for a 5251 terminal. This adds some value to the converter as those terminals are much easier to get than a working 5251.

The combination of the 5160 PC and 5250 emulation card is, let's say, slow as a rheumatic snail compared with the real 5251 terminal, but it's a nice thing to get it working.

5160.png (2.4 MiB) Viewed 46492 times

Having tested succesfully the converter with another system and now with support for multiple terminals I'm now pimping the script to publish it. I also plan to get the final PCB soon ready.

[Pingmeide]

Yes, this is really an awesome project - this is a piece of hardware I am looking for for ages.....
Those Twinax Terminals are now quite old, but given the quality of the IBM Hardware they are mostly in good shape ! (albeit sometimes a bit yellowed...)

You wrote "but I suppose that people that could be interested in replicating this can be counted off the fingers of one hand" - ah, I don't think so - I am sure there are quite a number of (Linux / Unix)-users in the world that are keen in using old IBM Terminals on their machine(s). They just need to know that this interface is available !

So if you decide to build a small batch of this interface (as a "plug-and-play"-solution, just for plugging it into the Hardware and the mains) I would be very interested in buying one. (Depends on the price you ask for, of course - so some price-information upfront would be very much appreciated before you start....)

Congratulations for all the effort in making this interface a reality !

Regards

Markus
Cologne / Germany

[inmbolmie]

So if you decide to build a small batch of this interface (as a "plug-and-play"-solution, just for plugging it into the Hardware and the mains) I would be very interested in buying one. (Depends on the price you ask for, of course - so some price-information upfront would be very much appreciated before you start....)

Hi Markus, I didn't make any plans to sell this "commercially" or anything remotely similar, but I can make a couple of boards just for fun. Price would be cost of materials plus shipping. This is not specially cheap though as it has more components than the typical cheap converter, and I have to buy them at retail price with VAT.

I have now a tested prototype PCB in a box, with a few improvements over the original design. The most important change is that I've tested the converter leaving out the +-12 voltages, as it seems to work just fine using only +-5 vols in the comparator. Also using a good dc-to-dc regulated converter the linear regulators are no longer needed and power efficiency is much improved. I've measured the consumption of the whole board at 250 milliamps @5V (100 for the Teensy plus 150 for the rest of components), well within the USB 2.0 500 milliamps limit, so now the board is now powered from the USB port and no external power is needed.

pcb.png (2 MiB) Viewed 46287 times

Not the prettiest board ever but it works.

converter.png (1.61 MiB) Viewed 46287 times

To get an approximate cost at today's prices would be:

• Teensy 4, 31€ today at amazon.es
• Dc-to-Dc converter, 17,21€ today at RS components
• ABS Box, 13,27 € today at RS components
• LM361, 2,89€ at RS components
• SN75110/2 approx 1€ each
• 74HCT04/86 approx 0,50 each (?)
• Capacitors, resistors etc, 1€ for all
• Micro USB cable 3€ at amazon.es
• Optional db15 connector 1€
• "Homemade" PCB, 3€
• Twinax female connector 3€

Grand total for the BOM appox. 80€

The files to build it would be available in github though for anyone wanting to make its own board.

[Pingmeide]

Hi, this is great - and 80€ for the complete box ("plug-in ready") is not a problem - given the time, effort and "brain-power" you have invested into this project and that it is not "mass-produced"....

Regarding the operation - I read the documentation you provided in this document and as far as I understand there is some software that has to be loaded into the arduino/teensy at startup (You provided the github link). At some other place some python code was mentioned. Is this still required OR is it just required to upload the arduino software, start it and then the TWINAX-Terminal displays the input prompt, given the ubuntu box is configured to expect a Terminal at its USB-Connector.

BTW: Is this the correct connector at the cable ? I know this one from printers, for connection to a computer I normally use "Type A" but this looks like a "Type B" ???

BTW-2 : The "rheumatic snail" comparison - it brought tears to my eyes from laughing....

Thanks for the update !

Regards
Markus

[inmbolmie]

Regarding the operation - I read the documentation you provided in this document and as far as I understand there is some software that has to be loaded into the arduino/teensy at startup (You provided the github link). At some other place some python code was mentioned. Is this still required OR is it just required to upload the arduino software, start it and then the TWINAX-Terminal displays the input prompt, given the ubuntu box is configured to expect a Terminal at its USB-Connector.

The Python script (Python 3 BTW) is required to run all the time at the computer side because it manages all the terminal conversion code and the shell sessions. The operation is, plug the converter to the USB port and the terminal to the Twinax connector, and run the python script. If there is only one terminal at address 0 the prompt appears at the terminal without any additional parameters. If you have more terminals and/or configured in other adresses I supppose where will need to be specified in the command line. The application also provides a debugging command prompt where you can send commands or keystrokes to any terminal.

Also I assume you will have to configure in the Python script the scancode to key mappings table for your keyboard, because it varies from terminal to terminal and for different languages and I don't have the slightest idea of how to get it done automatically. That is required for the script to know what character to send to the shell for each received scancode.

BTW: Is this the correct connector at the cable ? I know this one from printers, for connection to a computer I normally use "Type A" but this looks like a "Type B" ???

In the image is seen from one side and can lead to some confusion, but it is a type A to plug into the computer.

BTW-2 : The "rheumatic snail" comparison - it brought tears to my eyes from laughing....

Yeah, it is really that slow, It can be only improved polling that terminal with a longer interval, I suppose every frame it receives generates an interrupt that is managed in software, but the problem should be the 5160 itself not the card or protocol, should work much better with a fastest computer. The combo XT+ISA card was after all the "poor man's" 5250 of its day.

The general speed for the real 5251 is today much improved since the first version (specially line scrolling) and is reasonably usable.

[Pingmeide]

Hi there !

Thanks for all the information - now I can see how this all works - but this doesn't matter - I am still interested and eventually we can speed up the python code over time - don't worry - the most vital point is that it works after all - albeit a bit slow as you mentioned.

Currently I am looking for information how one can fix the "problem" with the different keyboard layouts, eg. spanish vs. US vs. German - just as you mentioned.

I found heaps of information on this address:

===> http://bitsavers.informatik.uni-stuttgart.de/pdf/ibm/

It seems they have scanned in tons of genuine IBM Documentation and eventually there's some information you need to progress with this project.

I will continue searching and will "report" my findings this evening - perhaps we are lucky...

Regards
Markus

P.S. Regarding the keys this document might be of help :
http://bitsavers.informatik.uni-stuttga ... _May74.pdf


Regarding the localisation of the 5251 Terminal this Document:
http://bitsavers.informatik.uni-stuttga ... _Jul79.pdf

has more info on page 105 (and some pages before and after that). There are some jumpers that need to be set or removed to configure the 5251 for a specific country.

[Pingmeide]

Hi,
in the post dated 16.May 2020 14:37 there was this document mentioned : "5250 Information Display to System/36 and System/38 System Units Product Attachment Information". It seems that there is some demand for this document and there is no way to find it online. To assist the other readers here I have contacted IBM today to check for their archives and if they can send an (electronic / PDF) copy over to me. They have already replied that my request has been forwarded to the department in charge of the archives etc. As soon as I receive an answer and / or get hold of a copy I will report it here. Fingers crossed ! I have worked for an IBM partner some time ago (pSeries / RS/6000) and I am quite confident that we will receive an answer - if we are lucky with the document attached.

Update 2020-JUN-06 : Yesterday I took some time to search for the document and I found some other documentation belonging to 3rd-party Software where this specific 5250 related document was mentioned - but sadly no link to get hold of a copy... And - as mentioned - the name was updated to "AS/400, System /38 and System /36..."

[inmbolmie]

...I am quite confident that we will receive an answer - if we are lucky with the document attached.

It would be very nice to get that document, I have already read the rest of available documents that you mentioned. There are even several revisions of that document that include "AS/400" information in addition to System/36 and System/38. If that document is what appears to be it could make easier for eeguru to advance with his project, or even for this project include some more advanced conversions like making a IBM Infowindow terminal VT100 compatible.

[RshnDrgn]

Just want to say I am glad to see this project, I saved one of these 5251 terminals out of the junk pile at work and wanted to get it work. I would be interested in one of the boards if you do end up building them. I used one of these at work when I first started, but they soon were replaced.