Improving serializers. Adding serializers examples in the notebook

from composite_field.rest_framework_support import CompositeFieldSerializer

from rest_framework import serializers

from composite_field.rest_framework_support import CompositeFieldSerializer

from imagedb.models import Instrument, NispDetector, Astrometry, NispRawFrame

  class Meta:

    model = Astrometry

    exclude = ('ctype1_coordinateType',

               'ctype1_projectionType',

               'ctype2_coordinateType',

               'ctype2_projectionType')    

    exclude = [f.name for g in Astrometry._meta.get_fields() 

               if hasattr(g, 'subfields') 

               for f in g.subfields.values()] 

class NispDetectorSerializer(serializers.ModelSerializer):

  astrometry = AstrometrySerializer(read_only = True)

  astrometry = AstrometrySerializer()

  class Meta:

    model = NispDetector

  class Meta:

    model = NispRawFrame

    exclude = ('commandedPointing_rightAscension',

               'commandedPointing_declination',

               'commandedPointing_orientation',

               'imageType_category',

               'imageType_firstType',

               'imageType_secondType')

    exclude = [f.name for g in NispRawFrame._meta.get_fields() 

               if hasattr(g, 'subfields') 

               for f in g.subfields.values()]

    depth = 2

%% Cell type:markdown id: tags:

# Data insertion and retrieval with Django models

In the following we show some examples in order to perform insertions and retrievals of metadata

%% Cell type:code id: tags:

``` python

from imagedb.models import Instrument

instrument = Instrument.objects.get(instrumentName='NISP')

print(instrument.telescopeName)

```

%% Cell type:markdown id: tags:

## Creating new objects

In the following we will create a NispRawFrame object. Since the NispRawFrame must have also a DataCotainer (i.e. a file to reference to), we first create a DataContainer instance.

We can see also that the fields that we have defined as composite fields provide some synctactic sugar when we initialize them with simple dictionaries.

%% Cell type:code id: tags:

``` python

from imagedb.models import ImageType, Pointing, NispDetector, DataContainer, NispRawFrame

from datetime import datetime

dataFile = DataContainer(

  fileFormat = 'fits',

  formatIdentifier = 'le1.nisprawframe',

  formatVersion = '1.0',

  url = "http://ia2-owncloud.oats.inaf.it/fake/7ff2f203/data/EUC_LE1_NISP_53892-Y-1_20170712T155430.1Z_00.00.fits"

)

# We have to save the data container to the DB before assigning it to a NispRawFrame

dataFile.save()

image = NispRawFrame(exposureTime = 105,

                     imgNumber = 16,

                     naxis1 = 2040,

                     naxis2 = 2040,

                     imageType = {'category':'SCIENCE',

                                  'firstType':'OBJECT',

                                  'secondType':'STD'},

                     observationDateTime = datetime.strptime("2025-06-21T18:27:23.000001",

                                                             "%Y-%m-%dT%H:%M:%S.%f"),

                     observationId = 53892,

                     ditherNumber = 1,

                     instrument = instrument,

                     commandedPointing = {'rightAscension':8.48223045516,

                                          'declination':8.48223045516,

                                          'orientation':64.8793517547},

                     filterWheelPosition = "Y",

                     grismWheelPosition = "OPEN"

                    )

image.frameFile = dataFile

```

%% Cell type:markdown id: tags:

Before saving an object, its surrogate key is still empty, i.e. equal to None

%% Cell type:code id: tags:

``` python

print(image.id)

image.save()

print(image.id)

```

%% Cell type:code id: tags:

``` python

# We can start creating a detector

d11 = NispDetector(detectorId = "11", gain = 1.0, readoutNoise = 0.0, rawFrame = image)

d11.save()

# Or we can use the create() in order to create and save immediately the new object

NispDetector.objects.create(detectorId = "12", gain = 1.0, readoutNoise = 0.0, rawFrame = image)

# or we can create the detector starting from the NispRawFrame, using the reversed relationship,

# using again the create method

image.detectors.create(

  detectorId = "13",

  gain = 1.0,

  readoutNoise = 0.0

)

```

%% Cell type:markdown id: tags:

## Objects retrieval

To retrieve objects from your database, construct a **QuerySet** via a **Manager** on your model class.

A **QuerySet** represents a collection of objects from your database. It can have zero, one or many filters. Filters narrow down the query results based on the given parameters. In SQL terms, a **QuerySet** equates to a **SELECT** statement, and a **filter** is a limiting clause such as **WHERE** or **LIMIT**.

You get a **QuerySet** by using your model’s **Manager**. Each model has at least one Manager, and it’s called **objects** by default.

The simplest way to retrieve objects from a table is to get all of them. To do this, use the **all()** method on a **Manager**:

%% Cell type:code id: tags:

``` python

len(NispRawFrame.objects.all())

```

%% Cell type:markdown id: tags:

But usually we want to filter the results. For this purpose we can use the **filter** method, both provided by the **Manager** and the **QuerySet**

%% Cell type:code id: tags:

``` python

# Retrieving all frames with observation id 53877 and filter Y

# and ordering the results by the ditherNumber

result = NispRawFrame.objects.filter(observationId=53877,

                                     filterWheelPosition='Y').order_by('ditherNumber')

for obj in result:

    print(obj.observationId, obj.filterWheelPosition, obj.ditherNumber)

```

%% Cell type:markdown id: tags:

We can also limit the number of results of a **QuerySet**, using the Python array-slice syntax:

%% Cell type:code id: tags:

``` python

from datetime import datetime

# Retrieving all NISP raw frames with observation date and time

# greater then or equal to 2026-06-22T17:00

len(NispRawFrame.objects.filter(observationDateTime__gte=datetime(2025,6,22,17,0)))

```

%% Cell type:code id: tags:

``` python

# Now limiting the result to 5 items

len(NispRawFrame.objects.filter(observationDateTime__gte=datetime(2025,6,22,17,0))[:5])

```

%% Cell type:markdown id: tags:

Keyword argument queries – in filter(), etc. – are “AND”ed together. If you need to execute more complex queries (for example, queries with OR statements), you can use **Q objects**.

%% Cell type:code id: tags:

``` python

# Now retrieving all frames with observation ids 53877 and 54349 and filter H

from django.db.models import Q

result = NispRawFrame.objects.filter(Q(observationId=53877) | Q(observationId=54349),

                                     filterWheelPosition='H').order_by('observationId', 'ditherNumber')

for obj in result:

    print(obj.observationId, obj.filterWheelPosition, obj.ditherNumber)

```

%% Cell type:markdown id: tags:

We can also traverse relations

%% Cell type:code id: tags:

``` python

# Getting all NispDetectors whose frame observation id is 53877 or 54349 and

# filterWheelPosition is Y.

# Notice that here we also use the 'in' operator

result = NispDetector.objects.filter(rawFrame__observationId__in=[53877, 54349],

                                     rawFrame__filterWheelPosition='Y').order_by(

                                     'rawFrame__observationId', 'rawFrame__ditherNumber',

                                     'detectorId')

for obj in result:

    print(obj.rawFrame.observationId, obj.rawFrame.ditherNumber, obj.detectorId)

```

%% Cell type:code id: tags:

``` python

# OR we can find the NispRawFrame whose referenced file url contains "LE1_NISP_52926-J-2"

result = NispRawFrame.objects.filter(frameFile__url__contains="LE1_NISP_52926-J-2")

for obj in result:

    print(obj.observationId, obj.frameFile.url)

```

%% Cell type:markdown id: tags:

In order to retrive a single object, instead of using **filter** we can use the **get** method. This method returns one object and raise exceptions if no object is found or if multiple objects satisfy the query

%% Cell type:code id: tags:

``` python

obj = NispRawFrame.objects.get(observationId=53892)

# now we delete such object from the database

obj.delete()

```

%% Cell type:markdown id: tags:

## Serializers

Let's see some differences between the plain Django serializers and the ModelSerializer class provided by the Django REST framework.

The first example uses the Django core serializers

%% Cell type:code id: tags:

``` python

from django.core import serializers

data = serializers.serialize('json',NispRawFrame.objects.filter(observationId=53877,

                            filterWheelPosition='Y').order_by('ditherNumber'))

print(data)

```

%% Cell type:markdown id: tags:

The following example, instead, uses the Django REST framework ModelSerializer class. In particular, see the file imagedb/serializers.py

%% Cell type:code id: tags:

``` python

from imagedb.serializers import NispRawFrameSerializer

import json

frame = NispRawFrameSerializer(NispRawFrame.objects.get(id=1))

print(json.dumps(frame.data, indent=2))

```