Fase 2 templates: unificati focus.py e focus2.py

## Real bugs found by the Phase 0 audit (fix during their phase, not

just style)

- **`observation.py` vs `snapshot.py`** (templates): these silently

  diverged after the snapshot merge. `observation.py`'s `fih_params`

  is missing the `"imagetyp"` key (snapshot.py has it);

  `observation.py` assumes binning is always scalar (snapshot.py

  handles int-or-list); `observation.py` has no `_NO_FILTER_CAMERAS`

  guard (would try to drive a filter wheel on cam3/échelle, which has

  none); `observation.py` has dome-slew logic during the exposure loop

  that snapshot.py dropped. `bias.py`/`box.py`/`flat.py`/`skyflat.py`

  all delegate to `observation.py`, not `snapshot.py` — decide: unify

  on one (snapshot.py's shape looks more correct/current) and update

  the four callers, or explicitly confirm they need to stay separate

  and backport the fixes.

- **`focus.py` vs `focus2.py`** (templates): both declare `self.name =

  "focus"`. `focus2.py` is an unreferenced, more complete rewrite

  (own ASCII plotter replacing a `gnuplotlib` dependency, `is_moving`

  polling, NaN-safe fitting, restores `foc.position` on failure) that

  was apparently never swapped in — `defaults/focus.json` still points

  at `focus.py`. Decide whether to promote focus2.py and delete

  focus.py, or discard focus2.py.

- ~~**`observation.py` vs `snapshot.py`**~~ FIXED — unified on

  `snapshot.py`, see the phase-2 checklist below for details.

- ~~**`focus.py` vs `focus2.py`**~~ FIXED — promoted `focus2.py`, see

  the phase-2 checklist below for details.

- **`testsonoff.py`** (templates): entire file commented out, imports

  a device (`sof`) that doesn't exist. Delete.

- **`box.py`/`focus.py`/`focus2.py`/`testpause.py` import a `cam` that

  doesn't exist** (templates, found post-Phase-0 while resolving

  `observation.py`/`snapshot.py`): all four do `from ..devices import

- ~~**`box.py`/`focus.py`/`focus2.py`/`testpause.py` import a `cam`

  that doesn't exist**~~ (templates, found post-Phase-0 while resolving

  `observation.py`/`snapshot.py`): all four did `from ..devices import

  cam`, but `devices.ini` only defines `cam1`/`cam2`/`cam3` sections

  (`noctua/devices/__init__.py`'s `dynamic_import` sets each device as

  an attribute named after its `devices.ini` section, so the module

  only ever gets `cam1`/`cam2`/`cam3`, never a bare `cam`) — importing

  any of these four files raises `ImportError` today. `focus.py`/

  `focus2.py` are in scope for this phase-2 pass (see below) and need

  this fixed as part of resolving which one to keep; `box.py`/

  `testpause.py` are deferred along with the rest of their files.

  any of these four files raised `ImportError`. FIXED in `focus.py`

  (now `focus2.py`'s promoted content, see below) and, minimally (one

  import line only, rest of the file still deferred), in `box.py` —

  `focus.py` imports `Box` from it at module load, so it had to be

  fixed too for `focus.py` to import at all. `testpause.py` still has

  the bug, deferred with the rest of that file.

- **`domotics.py`** (devices): `get()`/`put()` (lines 25-26, 58-59)

  start with a bare `return` before any body — every property built on

  them is silently non-functional. Currently not instantiated

      gitignored editor backup, not real source) so dropping that dead

      block with the rest of `observation.py` is not a behavior change

      for any current caller.

- [ ] Resolve `focus.py` vs `focus2.py` (see above). In scope.

- [x] Resolve `focus.py` vs `focus2.py` (see above). In scope.

      Promoted `focus2.py`: old `focus.py` (gnuplotlib-based) deleted,

      `focus2.py` renamed to `focus.py` (`defaults/focus.json` already

      pointed at the module name `"focus"`, unaffected). Fixed the

      `from ..devices import cam` bug (see above) in the promoted file

      by importing `cam1 as cam`. Stripped the leftover "Corrected

      relative import"/"No longer needed" artifact comments from an

      earlier automated pass. `pyproject.toml`'s `gnuplotlib` dependency

      comment updated — it said "If focus.py" (now gone) but the

      dependency is still needed for `utils/analysis.py:fit_star`'s

      optional `display=True` path.

      Also fixed **only** the same `cam` import line in `box.py` (one

      line: `cam1 as cam`) since `focus.py` imports `Box` from it at

      module load — leaving it broken would have made the promoted

      `focus.py` unimportable too. Rest of `box.py` stays deferred as

      scoped.

- [ ] `fillheader.py:212-217` swallows FITS-writing errors silently

      (logs only, no `self.error`, no `return`, falls through to save

      anyway) — align with the standard error pattern. In scope.

# System modules

# Other templates

from ..devices import cam

from ..devices import cam1 as cam

from ..utils.logger import log

from .basetemplate import BaseTemplate

from .snapshot import Template as Snapshot

from time import sleep

# Third-party modules

import gnuplotlib as gp

import numpy as np

from astropy.io import fits

from astropy.time import Time

# Other templates

from ..config.constants import pixscale, temp_fits

from ..devices import cam, foc

from ..devices import cam1 as cam, foc

from ..utils.analysis import fit_star

from ..utils.logger import log

from ..utils.structure import foc_path

from .box import Template as Box

def simple_ascii_plot(

        x_values,

        y_values,

        x_label="X",

        y_label="Y",

        title="Plot",

        width=50,

        height=15):

    """

    Generates a simple ASCII plot for the terminal.

    Scales y_values to fit within the height.

    """

    if not len(x_values) or not len(

            y_values) or len(x_values) != len(y_values):

        log.warning("ASCII Plot: Invalid or empty data for plotting.")

        return ""

    plot_str = []

    plot_str.append(f"{title:^{width}}")

    plot_str.append("")  # Empty line

    min_y, max_y = min(y_values), max(y_values)

    y_range = max_y - min_y

    if y_range == 0:  # Avoid division by zero if all y values are the same

        y_range = 1.0

    # Create the plot grid (transposed for easier filling)

    grid = [[' ' for _ in range(width)] for _ in range(height)]

    # Y-axis labels and scale

    y_axis_label_width = 8  # max width for y-axis numbers

    plot_width_effective = width - y_axis_label_width - 1  # -1 for the axis itself

    # Determine x positions (scaled to fit plot_width_effective)

    # This simple version assumes x_values are somewhat evenly spaced or just for labeling

    # For true x-axis scaling, it's more complex for ASCII.

    # We will place points at somewhat regular intervals across the width.

    num_points = len(x_values)

    if num_points == 0:

        return ""

    # Map y_values to plot height

    scaled_y = [int(((y - min_y) / y_range) * (height - 1)) for y in y_values]

    # Create a simple bar-like representation for each point

    # This is a very basic representation.

    # We will map each x_value to a column if possible, or group them.

    # For this simple plot, we'll iterate through y_values and represent them as bars

    # at distinct x positions if possible.

    # Let's make a bar chart where each x_value has its own "bar"

    # The width of each bar will be plot_width_effective / num_points

    bar_segment_width = max(1, plot_width_effective // num_points)

    for i in range(height):

        row_str = ""

        # Y-axis label

        y_val_at_row = max_y - \

            (i * y_range / (height - 1 if height > 1 else 1))

        row_str += f"{y_val_at_row:<{y_axis_label_width}.2f}|"

        for j in range(num_points):

            # Check if the current row corresponds to the bar height for this point

            # scaled_y is 0 at min_y, height-1 at max_y.

            # Grid is top-down, so we need to invert.

            if (height - 1 - scaled_y[j]) <= i:

                row_str += "*" * bar_segment_width

            else:

                row_str += " " * bar_segment_width

        plot_str.append(row_str.rstrip())

    # X-axis (very simplified)

    plot_str.append(

        "-" *

        y_axis_label_width +

        "+" +

        "-" *

        (plot_width_effective))

    # X-axis labels (approximate positions)

    # This is tricky to align perfectly in ASCII.

    # We'll just show min and max x for simplicity or label a few points.

    if num_points > 0:

        x_labels_line = " " * (y_axis_label_width + 1)

        if num_points == 1:

            x_labels_line += f"{x_values[0]:^{plot_width_effective}.1f}"

        elif num_points > 1:

            first_x_label = f"{x_values[0]:.1f}"

            last_x_label = f"{x_values[-1]:.1f}"

            middle_space = plot_width_effective - \

                len(first_x_label) - len(last_x_label)

            if middle_space < 0:

                middle_space = 0

            x_labels_line += first_x_label + " " * middle_space + last_x_label

        plot_str.append(x_labels_line)

        plot_str.append(f"{x_label:^{width}}")

    plot_str.append("")

    plot_str.append(f"{y_label} vs {x_label}")

    return "\n".join(plot_str)

class Template(BaseTemplate):

    def __init__(self):

        super().__init__()

        # Getting initial focus

        original_foc = foc.position

        if original_foc is None:

            log.error("Could not get initial focus position. Aborting focus.")

            self.error.append("Initial focus position is None.")

            return

        log.info(f"Initial focus position: {original_foc} µm")

        # starting from half before

        total_lenght = step * repeat

        foc.position -= total_lenght / 2

        # (repeat-1) steps span 'repeat' points, symmetric around the start

        total_length_focus_scan = step * (repeat - 1)

        start_focus = original_foc - total_length_focus_scan / 2

        log.info(

            f"Starting focus scan from {

                start_focus:.0f} µm to {

                start_focus +

                total_length_focus_scan:.0f} µm in {repeat} steps of {step} µm.")

        # Move to the start of the focus scan

        foc.position = int(round(start_focus))

        # Add a small delay for the focuser to settle if it's a physical device

        sleep(1.0)  # Adjust as needed, or check foc.is_moving if available

        # Preparing empty arrays for values

        m2_arr = np.array([], dtype=int)

        fwhm_arr = np.array([])

        m2_arr = np.array([], dtype=float)  # Use float for focus positions

        fwhm_arr = np.array([], dtype=float)

        self.output = []

        # Preparing the focus file path

        now = Time.now().isot

        file_path = foc_path(now)

        now_time = Time.now().isot

        # Ensure foc_path is from ..utils.structure

        file_path = foc_path(now_time)

        # Initial comment about variables a, b, c

        comment1 = f"Focus procedure of {now}"

        comment2 = f"rep m2[µm] fwhm[px]"

        # Initial comment about variables

        comment1 = f"Focus procedure of {now_time} for object '{objname}' with filter '{filt}'"

        comment2 = f"Step Focus[µm] FWHM[arcsec] Peak[ADU] BG[ADU] X[px] Y[px]"

        # Write the initial comment if the file doesn't exist

        with open(file_path, "a+") as file:

            log.info(f"Init file {file_path}")

            log.info(f"Using focus data file: {file_path}")

            file.seek(0)

            if not file.read(1):

                file.write(f"# {comment1}\n")

                file.write(f"# {comment2}\n")

        # Instanciating a Box template

        box = Box()

        box_template = Box()

        for rep_idx in range(repeat):

            current_focus_target = int(round(start_focus + rep_idx * step))

            log.info(

                f"Focus step {

                    rep_idx + 1}/{repeat} :: Target M2 Focus: {current_focus_target} µm")

            # Set focus position for this step

            foc.position = current_focus_target

            # Check if focuser is moving (if available) or sleep

            # This is important to ensure the image is taken at the correct

            # focus

            if hasattr(foc, 'is_moving'):

                while foc.is_moving:

                    log.debug("Waiting for focuser to settle...")

                    sleep(0.2)  # Short sleep while checking

                    if self.check_pause_or_abort():

                        return

            else:

                sleep(1.0)  # Generic delay if is_moving is not available

        for rep in range(0, repeat):

            log.info(f"Step {rep} of {repeat - 1}")

            # Get actual focus position after movement

            actual_m2_pos = foc.position

            if actual_m2_pos is None:

                log.warning(

                    f"Could not read M2 position at step {

                        rep_idx + 1}. Skipping this point.")

                continue

            m2_arr = np.append(m2_arr, actual_m2_pos)

            ################################

            ##### Taking a boxed image #####

            ################################

            # Update exptime and filter in params for the box template, if they

            # can change

            params["exptime"] = exptime

            params["filter"] = filt

            params["binning"] = binning

            box_template.run(params)  # saves temp.fits boxed image

            if self.check_pause_or_abort():

                return

            box.run(params)  # saves temp.fits boxed image

            log.debug(f"Getting {temp_fits}")

            try:

                data = fits.getdata(temp_fits)

            except FileNotFoundError:

                log.error(

                    f"Focus: {temp_fits} not found after box exposure. Skipping point.")

                # Add NaN for missing FWHM

                fwhm_arr = np.append(fwhm_arr, np.nan)

                continue

            ###############################

            ##### Fitting a 2d Moffat #####

            ###############################

            current_fwhm_arcsec = np.nan  # Default to NaN

            peak_adu, bg_adu, fit_x, fit_y = np.nan, np.nan, np.nan, np.nan

            log.debug(f"Fitting data")                        

            fitted = fit_star(data, model="moffat")

            ##############################

            ##### Filling the arrays #####

            ##############################

            m2 = foc.position

            fwhm = fitted.fwhm * pixscale * binning

            m2_arr = np.append(m2_arr, m2)

            fwhm_arr = np.append(fwhm_arr, fwhm)

            y, x = np.indices(data.shape)

            self.output = {

                "focus": {

                    "m2": m2_arr.tolist(),

                    "fwhm": fwhm_arr.tolist(),

                },

                # "data": data.tolist(),

                # "fit": fitted(x,y).tolist(),

                # "residuals": np.round( data - fitted(x,y) , 1).tolist(),

            try:

                # Estimate background from corners if possible, or use a fixed guess

                # For simplicity, using a fixed guess for now

                background_estimation = np.median(

                    data)  # A slightly better guess

                fitted = fit_star(

                    data,

                    model="moffat",

                    background_estimation=background_estimation)

                if fitted and hasattr(

                        fitted,

                        'fwhm') and hasattr(

                        fitted,

                        'xc'):

                    current_fwhm_arcsec = fitted.fwhm * pixscale * binning  # FWHM in arcsec

                    peak_adu = fitted.peak

                    bg_adu = fitted.background

                    fit_x = fitted.xc

                    fit_y = fitted.yc

                    log.info(

                        f"  -> Fit: FWHM={

                            current_fwhm_arcsec:.2f}\", Peak={

                            peak_adu:.0f}, BG={

                            bg_adu:.0f}, X={

                            fit_x:.1f}, Y={

                            fit_y:.1f}")

                else:

                    log.warning(

                        "  -> Fit failed or did not return expected attributes.")

            except Exception as e:

                log.warning(

                    f"  -> Star fitting error at focus {actual_m2_pos} µm: {e}")

            fwhm_arr = np.append(fwhm_arr, current_fwhm_arcsec)

            self.output = {  # Update self.output for potential real-time display/API

                "focus_run": {

                    "m2_positions": m2_arr.tolist(),

                    # Replace NaN for JSON

                    "fwhm_arcsec": np.nan_to_num(fwhm_arr, nan=-1.0).tolist(),

                    "current_point": {

                        "m2": actual_m2_pos,

                        "fwhm": current_fwhm_arcsec if not np.isnan(current_fwhm_arcsec) else -1.0,

                        "peak": peak_adu if not np.isnan(peak_adu) else -1.0,

                        "background": bg_adu if not np.isnan(bg_adu) else -1.0,

                        "fit_x_px": fit_x if not np.isnan(fit_x) else -1.0,

                        "fit_y_px": fit_y if not np.isnan(fit_y) else -1.0,

                    }

                }

            }

            # log.debug(self.output)

            # log.info(f"Found: focus={m2}µm, FWHM={fwhm}'' ")

            # opening all the time and not at begginning just for safety

            log.debug(f"Opening {file_path}")

            with open(file_path, "a") as file:

                msg = f"{rep:>3} {m2:>10} {fwhm:>10} "

                file.write(msg + "\n")

                log.info(f"Writing in focus file: rep={rep}, m2={m2}, fwhm={fwhm}")

            ######################

            ##### Moving M2  #####

            ######################

            foc.position += step

            log.warning(f"Moving focus by {step}µm, to {round(foc.position, 2)}µm")

            sleep(0.3)

                line_data = f"{

                    rep_idx +

                    1:<4d} {

                    actual_m2_pos:<10.1f} {

                    current_fwhm_arcsec:<10.2f} {

                    peak_adu:<9.0f} {

                    bg_adu:<7.0f} {

                        fit_x:<7.1f} {

                            fit_y:<7.1f}"

                file.write(line_data + "\n")

            if self.check_pause_or_abort():

                return

        ##############################

        ##### Showing the result #####

        ##############################

        log.info(f"focus={m2_arr}")

        log.info(f"fwhm={fwhm_arr}")

        gp.plot((m2_arr, fwhm_arr),

                unset="grid",

                _with="linespoints",

                terminal='dumb 90 30',

                xlabel="x=focus [µm], y=fwhm ['']")

        #################################

        ##### Back to initial focus #####

        #################################

        log.info("Setting back original focus")

        log.info("Focus scan complete. Results:")

        for i in range(len(m2_arr)):

            log.info(f"  Focus: {m2_arr[i]:.1f} µm, FWHM: {fwhm_arr[i]:.2f}\"")

        # Generate and print ASCII plot

        if len(m2_arr) > 1 and len(fwhm_arr) > 1:

            # Filter out NaN values for plotting and finding minimum

            valid_indices = ~np.isnan(fwhm_arr)

            plot_m2 = m2_arr[valid_indices]

            plot_fwhm = fwhm_arr[valid_indices]

            if len(plot_m2) > 0:

                ascii_chart = simple_ascii_plot(

                    plot_m2, plot_fwhm,

                    x_label="Focus M2 (µm)", y_label="FWHM (arcsec)",

                    title=f"Focus Curve - {objname} ({filt})",

                    width=80, height=20

                )

                log.info("\n" + ascii_chart)

                # Find best focus

                if len(plot_fwhm) > 0:

                    min_fwhm_idx = np.argmin(plot_fwhm)

                    best_focus_val = plot_m2[min_fwhm_idx]

                    best_fwhm_val = plot_fwhm[min_fwhm_idx]

                    log.info(

                        f"Optimal focus found at {

                            best_focus_val:.1f} µm with FWHM {

                            best_fwhm_val:.2f}\".")

                    log.info(

                        f"Moving M2 to optimal focus: {

                            best_focus_val:.0f} µm.")

                    foc.position = int(round(best_focus_val))

                else:

                    log.warning(

                        "No valid FWHM data points to determine optimal focus. Restoring original.")

                    foc.position = original_foc

            else:

                log.warning(

                    "No valid data points to plot or determine optimal focus. Restoring original.")

                foc.position = original_foc

        else:

            log.warning(

                "Not enough data points for a plot or to determine optimal focus. Restoring original focus.")

            foc.position = original_foc

        log.info(

            f"Focus procedure finished. M2 position set to: {

                foc.position} µm.")

    "numpy==2.3.5",

    "numba>=0.63.1",

    "loguru",

    "gnuplotlib", # If focus.py

    "gnuplotlib", # optional display=True path in utils/analysis.py:fit_star

    "pyvantagepro", # If meteo.py

#    "opencv-python", # If stream_opencv() in mako.py

    "quart",