Bantam CNC - final homing and backlash optimisation (phew) - index homing no more.

There's still some work to be done on this homing accuracy business. Let's see if we can crack it finally. The decision to use the Claude Code extension within Visual Studio Code has certainly allowed me to do stuff I'd never have been able to figure out myself, it's true - but the experience is pretty trying at times. If Claude has a personality it's that of an excitable junior engineer - tends to sound very pleased with itself and jumps to conclusions that aren't backed up with evidence. I guess we can hope that Claude will develop over the coming years to the point that it is a lot more dependable and less fallible.

Let's look at the homing moves. It seems to me that the movements are rather jerky, whereas a servo system should be capable of smooth, quiet movement. Let's log some moves and see if there's anything to be learned.

Homing transient analysis

All 10 index-clear events from the final switch position, velocity-derived from the raw position samples and aligned to each cycle's own capture instant (t=0, dashed green). Commanded (blue) is what the trajectory planner is asking for; feedback (orange) is what the axis is actually doing. Faint lines are individual cycles; overlap (or lack of it) at t=0 shows how consistently the capture lands on the velocity profile.

This doesn't look ideal - what's all that sinusoidal wobble about? Looks like marginal stability to me.

Here's the same data plotted as distance rather than velocity. Velocity is simply the derivative of distance.

±1mm step transient (no homing involved)

Let's do some transient testing. Here's the g code for a backlash test. The moves are G91 (relative / incremental, and this is a lathe, remember), which allows me to position the tool anywhere within the machine limits before running the test - that way I can check for backlash at different positions of the ballscrew. The 1mm move each way should be enough to break any stiction - beyond that, there's little to be gained in moving further before reversing back to the starting point.

(X axis backlash check)

(Jogs X +/-1mm from wherever it currently sits, pausing 1s each time it)

(returns to the start point -- alternating which direction it approaches)

(from, so a DTI held at that point shows the gap between a + and - approach.)

(Runs purely relative to current position -- no work offset needed.)

(edit these two to taste)

#<feed>   = 200   (mm/min)

#<cycles> = 10    (number of +/- pairs)

G21   (metric)

G94   (feed rate in units/min, not units/rev)

G91   (incremental mode -- all moves relative to current position)

o100 repeat [#<cycles>]

    G1 X1  F#<feed>     (jog +1mm)

    G1 X-1 F#<feed>     (return to start, approaching from +)

    G4 P1               (dwell 1s -- read DTI now: approach from +)

    G1 X-1 F#<feed>     (jog -1mm)

    G1 X1  F#<feed>     (return to start, approaching from -)

    G4 P1               (dwell 1s -- read DTI now: approach from -)

o100 endrepeat

G90   (back to absolute mode)

M2

All 10 dwell events from the z_backlash_check.ngc +/-1mm relative-jog test (no switch, no homing, no index involved at all -- a plain commanded stop), aligned to each move's own stop instant (t=0, dashed green). Commanded (blue) is what the trajectory planner is asking for; feedback (orange) is what the axis is actually doing. Faint lines are individual cycles. 

Nasty lumpy movement is still visible. Homing in on the transient portions of the +/-1mm backlash test, shows some unauthorised wobbling:

Cruise-phase velocity ripple

Z velocity during the constant-speed middle portion of a single +/-1mm move (200mm/min commanded) -- not the stop/settle transient, the actual translation. Commanded (blue) should be a flat line; feedback (orange) swings from near zero to more than double the commanded speed, cyclically, roughly every 40ms (~24-25Hz). 

Fundamental frequency is ~26Hz. Is this caused by the Linuxcnc controller or is it caused by the Lichuan servo driver? That's worth establishing before trying to optimise the wrong PID loop. IIRC, the Lichuan servo driver is actually pretty well set up for the matching motor.

Ripple: LinuxCNC's own commanded output vs. actual

Same +/-1mm cruise-phase window as before, but now including pid.z.output -- the actual velocity command LinuxCNC's PID sends downstream to the stepgen, before it ever reaches the drive. It shows the same ~26Hz ripple as the feedback, and at far larger amplitude than the nominal -3.33mm/s command -- meaning LinuxCNC's own loop output is actively oscillating, not just passing through a clean signal that gets corrupted later.

Setting D=0 in the INI file makes quite a difference:

Ripple fixed: pid.z.Dgain set to 0

Same +/-1mm cruise-phase window and signals as the previous capture, but with pid.z.Dgain set to 0 (was 3). pid.z.output now ramps smoothly and holds steady instead of oscillating between roughly -17 and +14mm/s -- suggests that the ~26Hz ripple was mostly the D-term amplifying quantization noise from the 5µm scale, not a real mechanical resonance.

Looks to me like too much Kp term. Reduce Kp to 35 from 50 and Kd to 0 from 3, I unchanged.

PID tuning progress

pid.z.output (the velocity command LinuxCNC sends to the stepgen) across three tuning stages, same +/-1mm cruise window each time. This is the clearest single signal for the whole investigation -- how hard the loop is working to hold a supposedly constant speed.

That is starting to look a lot better - the wobble is almost certainly due to too much Kp and Kd terms, resulting in overshoot and marginal stability.

PID tuning progress - Z axis

pid.z.output (the velocity command LinuxCNC sends to the stepgen) across three tuning stages, same +/-1mm cruise window each time. This is the clearest single signal for the whole investigation -- how hard the loop is working to hold a supposedly constant speed.

Looking a lot better now:

Let's look at the X axis now:

PID tuning progress - X axis

pid.x.output across the X-axis tuning sequence -- same diagnostic process as Z, but a different root cause: X's weak feedforward (FF1=0.4) meant Kp was carrying too much of the tracking burden, so raising FF1 (not cutting Kp or Kd) was the fix here.

Good - so we now seem to have a decent transient response on both axes. How has that improved the homing accuracy, if at all?

Servo homing results, Z and X

Each dot is one homing cycle's resting position (raw linear-scale feedback, not the DRO -- unaffected by the homing offset reset), shown as deviation from that group's own mean so spreads are directly comparable across groups measured in different sessions. Same jog-away and settle protocol throughout.

The Z axis is looking pretty good:

But the X axis is still pretty crap, even if it has improved a bit:

Final X axis backlash tests:

PID tuning progress - X axis

Clean before/after: original X gains (P=60 I=40 D=1 FF1=0.4) vs. the final tuned config (P=100 I=40 D=0 FF1=0.85). Mean peak position error went from ~41µm to 5.0µm (one encoder increment) on every single cycle.

And as for the X homing - has that finally improved? I'm close to losing the will to live here but I want to get this thing cracked.

Servo homing results, Z and X

Each dot is one homing cycle's resting position (raw linear-scale feedback, not the DRO -- unaffected by the homing offset reset), shown as deviation from that group's own mean so spreads are directly comparable across groups measured in different sessions. Same jog-away and settle protocol throughout.

Woohoo - finally looking half decent. I need to convince myself the effort was worth it - perhaps that conviction will come with time.

Note that these X distances are radius, not diameter. So the diameter home position error is 4x that of the Z axis ie +/-20um (0.8 thou). Hmm, that's shit. What more can be done?

One key difference is that the Z axis is still using index mark. we don't know how it would behave without that, as many changes happened since we implemented it. Is the x axis just as good as the z would be without index? In other words, has all this messing about with index marks been a waste of time?

Let's compare index mark homing vs the original proximity switch to double check....

Z axis homing - with index and without:

Conclusion: using the index mark from the servo driver makes sod all difference. So I had no need to bugger about with connections from the servo driver to the 7i85 interface and all the buggerage that it entailed. Instead, careful setup of the homing and PID parameters delivered the same results.

Final numbers:

• X axis homing accuracy - typically +/- 5-10um on radius (+/- 20um on diameter). 
• Z axis homing accuracy - typically +/- 5-10um.

And here's the proof of the homing accuracy (actually repeatability) for the X axis, typically within 5um after settling (10um on diameter):

And the backlash tests for the X axis (typically 15um variation):

Homing Z axis:

Backlash on Z axis:

Sod it. That may have to do for now unless I can be arsed to repeat the homing and backlash videos for the Z axis. Only time will tell.

And no, I'm not going to be using "index mark" homing on the Z axis. I've removed the relevant code from the HAL and INI files. And I never implemented index mark homing on the X axis, so no change required there.

Measuring and optimising the homing setup - proximity switch vs servo index mark

As mentioned in the last post, I'm not convinced the  homing setup on the Colchester bantam CNC is quite optimised. I'm thinking it might be helpful to connect up the servo driver synthesised encoder output to one of the spare 7i85 encoder inputs. But first, let's see what we are dealing with. I have to get an idea how much backlash there is in the system as well as optimising the actual homing setup. No point trying to get precise homing if the backlash is all over the place.

I ran a backlash test, where the carriage

• moves 1mm away from starting position in Z
• returns to starting position, dwells for 1 second (read position on encoder)
• moves 1mm in other direction
• returns to starting position, dwells for 1 second (reread position on encoder and compare)
• ...repeats for a total of 10 times

The difference between the 2 readings in each cycle is the backlash of the (unloaded) axis. Obviously this will be worse when there is a cutting load on the tool and there will also be some compliance (springiness) associated with the applied load. 

Here are the results:

Z axis settling trace

Commanded vs. actual feedback position from hm2_5i25.0.encoder.03 during the ±1mm backlash-check program, sampled at the 3ms servo period via sampler/halsampler. Ten cycles shown below; the shaded band marks the ±4.5µm PID deadband.

Peak overshoot (mean)

28.5µm

Peak overshoot (max)

70.0µm

Settled error (mean)

3.5µm

Dwells sampled

20

Per-dwell readings

Peak deviation during each 1s hold, and the error remaining once the dwell ends

#	t (s)	peak dev (µm)	settled err (µm)
1	0.00	+15.0	0.0
2	1.65	-20.0	0.0
3	3.29	-50.0	+5.0
4	4.93	+25.0	+5.0
5	6.57	-70.0	+10.0
6	8.21	+40.0	+5.0
7	9.85	-45.0	0.0
8	11.49	+40.0	0.0
9	13.13	-15.0	+5.0
10	14.77	+30.0	+5.0
11	16.41	+10.0	0.0
12	18.05	+20.0	-5.0
13	19.69	-35.0	-5.0
14	21.33	+40.0	-5.0
15	22.97	-15.0	-5.0
16	24.61	+20.0	0.0
17	26.25	-10.0	-5.0
18	27.89	+30.0	0.0
19	29.53	-20.0	-5.0
20	31.17	+20.0	-5.0

Captured from a live run of z_backlash_check.ngc (10 cycles, 200mm/min, 1s dwells). Servo period 3ms. Raw samples: z_trajectory.txt.

Now for the actual homing tests. Logging the raw encoder position (which doesn't vary between homing events) shows the variation between homing events.

At the instant home_sw_out falls (ie when the index mark is captured) across 12 capture events from 6 homing cycles:

t(s)	fb (mm)	cmd (mm)	following error (µm)
27.201	3.0150	2.9538	61.2
29.901	3.0100	2.9658	44.2
35.874	3.0050	2.9588	46.2
38.565	3.0200	2.9618	58.2
44.451	3.0150	2.9838	31.2
47.121	3.0000	2.9658	34.2
52.608	3.0200	2.9638	56.2
55.305	3.0150	2.9728	42.2
60.975	2.9950	2.9638	31.2
63.654	3.0100	3.0088	1.2
68.799	3.0150	2.9938	21.2
71.457	3.0100	2.9998	10.2

Two observations:

1. The linear scale reading itself (fb) at the capture instant spans 2.9950–3.0200mm — a 25µm spread, even though the underlying index event is electrically identical every time.
2. The following error at that instant swings from 1.2µm to 61.2µm — because the capture happens mid-motion (during the backoff/re-approach), and how much the axis is lagging or overshooting its commanded trajectory at that precise 3ms tick varies a lot cycle to cycle.

The index event is perfectly repeatable, but LinuxCNC only ever sees "the signal changed" at ordinary software-polling resolution, and captures whatever the linear scale happens to read at that moving instant — which inherits the servo loop's in-motion dynamics rather than a settled, at-rest value. That 25µm scale-reading spread is a real, substantial chunk of the ~70µm total homing scatter we measured.

So the index-gating mechanism is working as designed, but although it can gate when the position gets read, it can't make that reading immune to whatever the servo loop is doing at that mid-motion moment. Fixing that further would mean a different approach entirely (e.g. capturing position only once the axis has fully stopped and settled after the index fires, rather than during the backoff move itself) - that level of work isn't going to happen.

After some buggering about with the PID values, the search distance and the homing velocity, the error is much better and it matches the DTI readings I've made by eye:

	mean |err|	max |err|	gap
Original switch position	24.0µm	56.2µm	~2119µm
Best prior (884µm gap)	10.7µm	56.2µm	~886µm
Current	11.4µm	28.8µm	~482µm

Matching the best mean we've seen, but with a tighter max (28.8 vs 56.2µm) and less than half the wasted travel. This lines up well with the DTI's 3-40µm read (our computed range is 1.0-28.8µm — DTI rounding easily accounts for the difference). We've gone from ~2.1mm of unnecessary travel and 24µm mean error down to ~0.5mm and ~11µm mean error.

At this point we have: index_toggle + mux_generic wiring validated, HOME_FINAL_VEL and HOME_LATCH_VEL tuned, the switch repositioned to a good phase alignment, and PID reverted to its best values. 

The current HAL/INI state uploaded to Github repo and the Claude project memory updated with the final settings and results.

• Committed and pushed (becc808 on feature/qt-lathe-macros): the index_toggle/mux_generic HAL wiring, HOME_FINAL_VEL/HOME_LATCH_VEL tuning, and the new index_toggle.comp source.
• Currently defaults to legacy switch homing on boot (mux-gen.00.sel-bit-00=FALSE) — flip to index-gated live with halcmd setp mux-gen.00.sel-bit-00 1 whenever you want it, or say the word if you'd like it made the permanent default.
• PID gains reverted to the original P=50/D=3 — the tuning experiments there didn't pan out.
• Final result: mean home-position error down to ~11µm from the original ~24-36µm, matching your DTI reading — driven mostly by HOME_FINAL_VEL and the switch repositioning, with index-gating itself turning out to be a smaller factor than expected once the real bottleneck (mid-motion capture during the software-polled switch/index transition) was understood.
• Memory's consolidated into a single clean record for next time, including the reusable debugging techniques (halrun dry-testing, sampler/halsampler, the shell-arrow gotcha) in case this comes up again.

Comparing both axes on the same basis (ie how does the index mark homing scheme compare wit the original proximity switch scheme?)

Z home repeatability, both schemes

Each dot is one homing cycle's resting hm2_5i25.0.encoder.03.position (the raw Z scale feedback, not the DRO -- unaffected by the homing offset reset). Jog-away and settle time were held constant between cycles; only the homing method changes.

I think we can honestly say that there is no discernible difference between the 2 methods. Hmm. Wiring the servo driver up to the 7i85 may well have been a waste of time.

https://claude.ai/code/artifact/be0d16d4-ab58-42b8-bfda-7fb55a812683

After some similar messing about (sorry, "optimising"), we have implemented the "learnings" from the Z axis activity over to the the X axis. I'd like to think we are better than when we started - and also close to the optimal point.

Now compare final results for X and Z axes:

Servo homing results, Z and X

Each dot is one homing cycle's resting position (raw linear-scale feedback, not the DRO -- unaffected by the homing offset reset), shown as deviation from that group's own mean so spreads are directly comparable across groups measured in different sessions. Same jog-away and settle protocol throughout.

Legacy: prox switch trigger. Index-gated: index_toggle HAL component arming the servo's hardware index pulse off the same switch. Same jog-away distance, same settle time, same HOME_FINAL_VEL for both.

Legacy: prox switch trigger. Index-gated: index_toggle HAL component arming the servo's hardware index pulse off the same switch. Same jog-away distance, same settle time, same HOME_FINAL_VEL for both.

Both charts show deviation from each group's own mean (in µm), so spreads are directly comparable — hover any dot for its exact reading and cycle number.

• Z: legacy vs. index-gated — 75µm vs 70µm spread, essentially a tie once HOME_FINAL_VEL was fixed
• X: before vs. after the fix — 120µm → 70µm spread, with the same improvement made.

I have to say there is still work to be done, as these are still shit numbers, given that we have a 5um resolution on the encoders. Let's leave it here until next time.....