Newer
Older
"# Data insertion and retrieval with Django models\n",
"In the following we show some examples in order to perform insertions and retrievals of metadata "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"from imagedb.models import Instrument\n",
"\n",
"instrument = Instrument.objects.get(instrumentName='NISP')\n",
"\n",
"print(instrument.telescopeName)\n"
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Creating new objects\n",
"\n",
"In the following we will create a NispRawFrame object. Since the NispRawFrame must have also a DataCotainer (i.e. a file to reference to), we first create a DataContainer instance.\n",
"\n",
"We can see also that the fields that we have defined as composite fields provide some synctactic sugar when we initialize them with simple dictionaries."
]
},
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from imagedb.models import ImageType, Pointing, NispDetector, DataContainer, NispRawFrame\n",
"\n",
"from datetime import datetime\n",
"\n",
"dataFile = DataContainer(\n",
" fileFormat = 'fits',\n",
" formatIdentifier = 'le1.nisprawframe',\n",
" formatVersion = '1.0',\n",
" url = \"http://ia2-owncloud.oats.inaf.it/fake/7ff2f203/data/EUC_LE1_NISP_53892-Y-1_20170712T155430.1Z_00.00.fits\"\n",
")\n",
"\n",
"# We have to save the data container to the DB before assigning it to a NispRawFrame\n",
"dataFile.save()\n",
"\n",
"image = NispRawFrame(exposureTime = 105,\n",
" imgNumber = 16,\n",
" naxis1 = 2040,\n",
" naxis2 = 2040,\n",
" imageType = {'category':'SCIENCE', \n",
" 'firstType':'OBJECT', \n",
" 'secondType':'STD'},\n",
" observationDateTime = datetime.strptime(\"2025-06-21T18:27:23.000001\", \n",
" \"%Y-%m-%dT%H:%M:%S.%f\"),\n",
" observationId = 53892,\n",
" ditherNumber = 1,\n",
" instrument = instrument,\n",
" commandedPointing = {'rightAscension':8.48223045516,\n",
" 'declination':8.48223045516,\n",
" filterWheelPosition = \"Y\",\n",
" grismWheelPosition = \"OPEN\"\n",
" )\n",
"\n",
"\n",
"\n",
"image.frameFile = dataFile"
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Before saving an object, its surrogate key is still empty, i.e. equal to None"
]
},
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"print(image.id)\n",
"image.save()\n",
"print(image.id)"
]
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"# We can start creating a detector\n",
"\n",
"d11 = NispDetector(detectorId = \"11\", gain = 1.0, readoutNoise = 0.0, rawFrame = image)\n",
"d11.save()\n",
"\n",
"# Or we can use the create() in order to create and save immediately the new object\n",
"NispDetector.objects.create(detectorId = \"12\", gain = 1.0, readoutNoise = 0.0, rawFrame = image)\n",
"\n",
"# or we can create the detector starting from the NispRawFrame, using the reversed relationship,\n",
"# using again the create method\n",
" gain = 1.0,\n",
" readoutNoise = 0.0\n",
")"
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Objects retrieval\n",
"\n",
"To retrieve objects from your database, construct a **QuerySet** via a **Manager** on your model class. \n",
"\n",
"A **QuerySet** represents a collection of objects from your database. It can have zero, one or many filters. Filters narrow down the query results based on the given parameters. In SQL terms, a **QuerySet** equates to a **SELECT** statement, and a **filter** is a limiting clause such as **WHERE** or **LIMIT**.\n",
"\n",
"You get a **QuerySet** by using your model’s **Manager**. Each model has at least one Manager, and it’s called **objects** by default.\n",
"\n",
"The simplest way to retrieve objects from a table is to get all of them. To do this, use the **all()** method on a **Manager**:"
]
},
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"len(NispRawFrame.objects.all())"
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"But usually we want to filter the results. For this purpose we can use the **filter** method, both provided by the **Manager** and the **QuerySet**\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Retrieving all frames with observation id 53877 and filter Y\n",
"# and ordering the results by the ditherNumber\n",
"\n",
"result = NispRawFrame.objects.filter(observationId=53877, \n",
" filterWheelPosition='Y').order_by('ditherNumber')\n",
"\n",
"for obj in result:\n",
" print(obj.observationId, obj.filterWheelPosition, obj.ditherNumber)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also limit the number of results of a **QuerySet**, using the Python array-slice syntax:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from datetime import datetime\n",
"\n",
"# Retrieving all NISP raw frames with observation date and time \n",
"# greater then or equal to 2026-06-22T17:00\n",
"len(NispRawFrame.objects.filter(observationDateTime__gte=datetime(2025,6,22,17,0)))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Now limiting the result to 5 items\n",
"\n",
"len(NispRawFrame.objects.filter(observationDateTime__gte=datetime(2025,6,22,17,0))[:5])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Keyword argument queries – in filter(), etc. – are “AND”ed together. If you need to execute more complex queries (for example, queries with OR statements), you can use **Q objects**."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Now retrieving all frames with observation ids 53877 and 54349 and filter H\n",
"from django.db.models import Q\n",
"\n",
"result = NispRawFrame.objects.filter(Q(observationId=53877) | Q(observationId=54349), \n",
" filterWheelPosition='H').order_by('observationId', 'ditherNumber')\n",
"\n",
"for obj in result:\n",
" print(obj.observationId, obj.filterWheelPosition, obj.ditherNumber)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also traverse relations"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Getting all NispDetectors whose frame observation id is 53877 or 54349 and \n",
"# filterWheelPosition is Y.\n",
"# Notice that here we also use the 'in' operator\n",
"\n",
"result = NispDetector.objects.filter(rawFrame__observationId__in=[53877, 54349],\n",
" rawFrame__filterWheelPosition='Y').order_by(\n",
" 'rawFrame__observationId', 'rawFrame__ditherNumber', \n",
" 'detectorId')\n",
"\n",
"for obj in result:\n",
" print(obj.rawFrame.observationId, obj.rawFrame.ditherNumber, obj.detectorId)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# OR we can find the NispRawFrame whose referenced file url contains \"LE1_NISP_52926-J-2\"\n",
"\n",
"result = NispRawFrame.objects.filter(frameFile__url__contains=\"LE1_NISP_52926-J-2\")\n",
"\n",
"for obj in result:\n",
" print(obj.observationId, obj.frameFile.url)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In order to retrive a single object, instead of using **filter** we can use the **get** method. This method returns one object and raise exceptions if no object is found or if multiple objects satisfy the query"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"obj = NispRawFrame.objects.get(observationId=53892)\n",
"\n",
"# now we delete such object from the database\n",
"\n",
"obj.delete()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Serializers\n",
"\n",
"Let's see some differences between the plain Django serializers and the ModelSerializer class provided by the Django REST framework.\n",
"The first example uses the Django core serializers"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
"source": [
"from django.core import serializers\n",
"\n",
"data = serializers.serialize('json',NispRawFrame.objects.filter(observationId=53877, \n",
" filterWheelPosition='Y').order_by('ditherNumber'))\n",
"\n",
"print(data)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The following example, instead, uses the Django REST framework ModelSerializer class. In particular, see the file imagedb/serializers.py"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from imagedb.serializers import NispRawFrameSerializer\n",
"import json\n",
"\n",
"frame = NispRawFrameSerializer(NispRawFrame.objects.get(id=1))\n",
"print(json.dumps(frame.data, indent=2))\n",
"\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Django Shell-Plus",
"language": "python",
"name": "django_extensions"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
}
},
"nbformat": 4,
"nbformat_minor": 2
}