I need to see a schematic or a block diagram at the very least. For the moment, if I understand this correctly, I can tell you that if adjusting the DAC you only have 6/10/15 counts, that is too low. It means that the DAC maximum voltage is too high.

The solution is to create an attenuator after the DAC, that will bring the DAC full range to the maximum voltage you need for calibration. That way, instead of 15 counts maximum, you will have the full DAC counts, 256 for and 8-bit DAC or 4096 for a 12-bit DAC.

If you are interested to continue this discussion answer here, and I will send you my email address.

Hi,
A broad maybe unanswerable wrong question from me but in hopes to find the solution or path.

Example:
I’m calibrating a 1.5kw 50ohm 208v generator and the DAC count is too low.
It is a cal/verify test.
My points are 6/10/15
I know its hard without seeing actual stuff,
I am looking to calibrate the counts in a front panel window, to the pots on a measurement board.
Is there a specific adjustment typically on the measurement board that boost the DAC count ?
Or would my analog, digital board or 186 or MB have something also?
Op amp? / gain?

I made gain adjustments on a 5.5kw 50ohm 400v and it worked fine.
But the problem is that changing the fwd gain, it had to be changed each time during test, but in command line passed like a charm.
Any idea why the EEPROM had to be kept re adjusting?

Hope I’m clear enough…

You need to consider the ADC or DAC resolution (or the number of bits), the reference voltage, what kind of digital interface (parallel, SPI, I2C), the signal-to-noise ratio. It really depends on your application. Any ADC/DAC datasheet will show you these parameters.

If the DAC is non-linear, in other words, the INL is high, a two point calibration is not enough. Depending on the DAC transfer function non-linearity, the calibration can be achieved with a few points and then interpolation, or with a look up table and interpolation.

More about this in a future article.