Skip to content

Commit a2adb59

Browse files
loco-philippeloco-philippe
committed
v1.2.0
1 parent b030248 commit a2adb59

File tree

4 files changed

+65
-206
lines changed

4 files changed

+65
-206
lines changed

‎documentation/release/1.x.rst

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,19 @@
11
Version 1.x
22
===========
33

4+
1.2.0 (2024-01-05)
5+
--------------------
6+
- **issues**
7+
- bugs #7, #8, #9, #10, #11, #12
8+
- use of relative Namespace in ntv_type
9+
10+
- **NTV Structure:**
11+
- update NTV_global_namespace
12+
- new management of Namespace files .ini
13+
- separation of NTVtype (attribute of NTV entity) and Datatype (data type used in NTVtype)
14+
- add `from_file` function for creating a custom Namespace and all associated Namespace and Datatype
15+
16+
417
1.1.0 (2023-11-23)
518
--------------------
619

‎json_ntv/namespace.py

Lines changed: 39 additions & 199 deletions
Original file line numberDiff line numberDiff line change
@@ -1,185 +1,18 @@
11
# -*- coding: utf-8 -*-
22
"""
3-
Created on Jan 20 2023
4-
53
@author: Philippe@loco-labs.io
64
75
The `namespace` module is part of the `NTV.json_ntv` package ([specification document](
8-
https://github.com/loco-philippe/NTV/blob/main/documentation/JSON-NTV-standard.pdf)).
9-
10-
It contains the `Namespace` and the `Datatype` classes and the `str_type` method for NTV entities.
11-
12-
# 0 - Presentation
13-
14-
The Datatype is defined by a name and a Namespace. The name is unique in the Namespace
15-
16-
A Namespace is represented by a string followed by a point.
17-
Namespaces may be nested (the global Namespace is represented by an empty string).
18-
19-
The Namespace representations are added to the value of an Datatype to have an absolute
20-
representation of an Datatype (long_name).
21-
22-
*Example for an absolute representation of an Datatype defined in two nested Namespace :*
23-
*“ns1.ns2.type”*
24-
*where:*
25-
- *ns1. is a Namespace defined in the global Namespace,*
26-
- *ns2. is a Namespace defined in the ns1. Namespace,*
27-
- *type is a Datatype defined in the ns2. Namespace*
28-
29-
# 1 - Global Namespace
30-
31-
The structure of types by namespace makes it possible to have types corresponding
32-
to recognized standards at the global level.
33-
Generic types can also be defined (calculation of the exact type when decoding the value).
34-
35-
The global namespace can include the following structures:
36-
37-
## 1.1 - Simple (JSON RFC8259)
38-
39-
| type (generic type)| value example |
40-
|--------------------|-------------------------------|
41-
| boolean (json) | true |
42-
| null (json) | null |
43-
| number (json) | 45.2 |
44-
| string (json) | "string" |
45-
| array (json) | [1, 2, 3] |
46-
| object (json) | { "str": "test", "bool": true}|
47-
48-
## 1.2 - Datation (ISO8601 and Posix)
49-
50-
| type (generic type)| value example |
51-
|--------------------|-------------------------------|
52-
| year | 1998 |
53-
| month | 10 |
54-
| day | 21 |
55-
| wday | 2 |
56-
| yday | 251 |
57-
| week | 38 |
58-
| hour | 20 |
59-
| minute | 18 |
60-
| second | 54 |
61-
| date (dat) | “2022-01-28” |
62-
| time (dat) | “T18:23:54”, “18:23”, “T18” |
63-
| datetime (dat) | “2022-01-28T18-23-54Z”, “2022-01-28T18-23-54+0400” |
64-
65-
## 1.3 - Duration (ISO8601 and Posix)
66-
67-
| type (generic type)| value example |
68-
|--------------------|-------------------------------|
69-
| period | "2007-03-01T13:00:00Z/2008-05-11T15:30:00Z" |
70-
| duration | "P0002-10- 15T10:30:20" |
71-
| timearray | [date1, date2] |
72-
73-
## 1.4 - Location (RFC7946 and Open Location Code):
74-
75-
| type (generic type) | value example |
76-
|---------------------|------------------------------|
77-
| point (loc) | [ 5.12, 45.256 ] (lon, lat) |
78-
| line (loc) | [ point1, point2, point3 ] |
79-
| ring | [ point1, point2, point3 ] |
80-
| multiline | [ line1, line2, line3] |
81-
| polygon (loc) | [ ring1, ring2, ring3] |
82-
| multipolygon (loc) | [ poly1, poly2, poly3 ] |
83-
| box (loc) | [ -10.0, -10.0, 10.0, 10.0 ] |
84-
| geojson (loc) | {“type”: “point”, “coordinates”: [40.0, 0.0] } |
85-
| codeolc (loc) | “8FW4V75V+8F6” |
86-
87-
## 1.5 - Tabular data
88-
89-
| Datatype | NTVvalue |
90-
|----------|--------------------------------------------------------|
91-
| row | JSON-array of JSON-NTV |
92-
| field | JSON-array of NTVvalue (following JSON-TAB format) |
93-
| table | JSON-array of JSON-NTV fields with the same length |
94-
95-
96-
## 1.6 - Normalized strings
97-
98-
The type could be `uri`, cf exemples :
99-
- "https://www.ietf.org/rfc/rfc3986.txt"
100-
- "https://gallica.bnf.fr/ark:/12148/bpt6k107371t"
101-
- "urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6"
102-
- "ni:///sha-256;UyaQV-Ev4rdLoHyJJWCi11OHfrYv9E1aGQAlMO2X_-Q"
103-
- "geo:13.4125,103.86673" *(RFC5870)*
104-
- "info:eu-repo/dai/nl/12345"
105-
- "mailto:John.Doe@example.com"
106-
- "news:comp.infosystems.www.servers.unix"
107-
- "urn:oasis:names:specification:docbook:dtd:xml:4.1.2"
108-
109-
## 1.7 - Namespaces
110-
111-
Namespaces could also be defined to reference for example:
112-
- geopolitical entities: ISO3166-1 country code (for example "fr." for France)
113-
- data catalogs, for example:
114-
115-
| Datatype | example JSON-NTV |
116-
|--------------|----------------------------------------------------------------------|
117-
| schemaorg. | <div>{ “:schemaorg.propertyID”: “NO2” }</div><div>{ “:schemaorg.unitText”:”µg/m3”}</div> |
118-
| darwincore. | { “:darwincore.acceptedNameUsage”: “Tamias minimus” } |
119-
120-
## 1.8 - Identifiers
121-
122-
For example :
123-
124-
| type | definition | exemple |
125-
|--------------|---------------------------------|-----------------------|
126-
| fr.uic | code UIC station | 8757449 |
127-
| fr.iata | code IATA airport | CDG |
128-
129-
130-
# 2 - Example of using a `fr.` namespace
131-
132-
This namespace is dedicated to datasets associated with the France geopolitical namespace
133-
(see also the [presentation document](
134-
https://github.com/loco-philippe/NTV/blob/main/documentation/JSON-NTV-namespace-fr.pdf)).
135-
136-
A namespace defines:
137-
- identifiers used to access additional data,
138-
- namespaces associated with catalogs or data sets,
139-
- structured entities used to facilitate the use of data
140-
141-
## 2.1 - Identifiers
142-
They could correspond to identifiers used in many referenced datasets
143-
(via a data schema or a data model).
144-
145-
For example :
146-
147-
| type | definition | example |
148-
|--------------|---------------------------------|-----------------------|
149-
| fr.dep | code département | 60 |
150-
| fr.cp | code postal | 76450 |
151-
| fr.naf | code NAF | 23 |
152-
| fr.siren | code SIREN enterprise | 418447363 |
153-
| fr.fantoir | code FANTOIR voie | 4500023086F |
154-
| fr.uai | code UAI établissement | 0951099D |
155-
| fr.aca | code académies | 22 |
156-
| fr.finessej | code FINESS entité juridique | 790001606 |
157-
| fr.rna | code WALDEC association | 843S0843004860 |
158-
| fr.spi | code SPI numéro fiscal | 1899582886173 |
159-
| fr.nir | code NIR sécurité sociale | 164026005705953 |
160-
161-
## 2.2 Namespaces
162-
Namespaces could correspond to catalogs or data sets whose data types are identified
163-
in data models or in referenced data schemas.
164-
165-
For example :
166-
167-
| type | example JSON-NTV |
168-
|-------------|-----------------------------------------------------------|
169-
| fr.sandre. | <div>{ ":fr.sandre.CdStationHydro": K163 3010 01 }</div><div>{ ":fr.sandre.TypStationHydro": "standard" }</div> |
170-
| fr.synop. | <div>{ ":fr.synop.numer_sta": 07130 }</div><div>{ ":fr.synop.t": 300, ":fr.synop.ff": 5 }</div> |
171-
| fr.IRVE. | <div>{ ":fr.IRVE.nom_station": "M2026" }</div><div>{ ":fr.IRVE.nom_operateur": "DEBELEC" }</div> |
172-
| fr.BAN. | <div>{ ":fr.BAN.numero": 54 }</div><div>{ ":fr.BAN.lon": 3.5124 }</div>|
173-
174-
## 2.3 Entities
175-
They could correspond to assemblies of data associated with a defined structure.
176-
177-
For example :
178-
179-
| type | example JSON-NTV |
180-
|--------------|----------------------------------------------------------|
181-
| fr.parcelle | <div>{“maParcelle:fr.parcelle”: [ 84500, 0, I, 97]}</div><div><i>(fr.cp, fr.cadastre.préfixe, fr.cadastre.section, fr.cadastre.numéro)</i></div> |
182-
| fr.adresse | <div>{“monAdresse:fr.adresse”: [ 54, bis, rue de la mairie, 78730 ]</div><div><i>(fr.BAN.numero, fr.BAN.rep, fr.BAN.nom_voie, fr.cp)</i></div> |
6+
https://loco-philippe.github.io/ES/JSON%20semantic%20format%20(JSON-NTV).htm)).
7+
8+
It contains the `Namespace`, `Datatype`, `DatatypeError` classes and
9+
the functions `agreg_type`, `from_file`, `relative_type` and `str_type`.
10+
11+
Namespace and Datatype entities are used to define NTVtype.
12+
13+
For more information, see the
14+
[user guide](https://loco-philippe.github.io/NTV/documentation/user_guide.html)
15+
or the [github repository](https://github.com/loco-philippe/NTV).
18316
18417
"""
18518
import configparser
@@ -215,17 +48,15 @@ def agreg_type(str_typ, def_type, single):
21548
return clas.add(_join_type(def_type.long_name, str_typ))
21649
raise DatatypeError(str_typ + ' and ' + def_type.long_name + ' are incompatible')
21750

218-
219-
def _join_type(namesp, str_typ):
220-
'''join Namespace string and Datatype or Namespace string'''
221-
namesp_split = namesp.split('.')[:-1]
222-
for name in str_typ.split('.'):
223-
if not name in namesp_split:
224-
namesp_split.append(name)
225-
return '.'.join(namesp_split)
226-
22751
def from_file(file, name, long_parent=None):
228-
'''create a new Ntvtype with all subtypes'''
52+
'''create a set of Datatype and Namespace associated to a custom Namespace
53+
54+
*Parameters*
55+
56+
- **file** : .ini file - description of the Datatype and Namespace
57+
- **name** : string - name of the root custom Namespace
58+
- **long_parent** : longname of the parent Namespace of the root Namespace
59+
'''
22960
long_parent = '' if not long_parent else long_parent
23061
if name[0] != '$':
23162
raise DatatypeError(name + ' is not a custom Datatype')
@@ -234,19 +65,8 @@ def from_file(file, name, long_parent=None):
23465
schema_nsp = Namespace(name, long_parent)
23566
config = configparser.ConfigParser()
23667
config.read(file)
237-
_add_file(config, schema_nsp)
68+
_add_namespace(config, schema_nsp)
23869

239-
def _add_file(config, namesp):
240-
if namesp.name in config.sections():
241-
confname = config[namesp.name]
242-
if 'namespace' in confname:
243-
for nspname in json.loads(confname['namespace']):
244-
nsp = Namespace(nspname, namesp, force=True)
245-
_add_file(config, nsp)
246-
if 'type' in confname:
247-
for typ in json.loads(confname['type']):
248-
Datatype(typ, namesp, force=True)
249-
25070
def relative_type(str_def, str_typ):
25171
'''return relative str_typ string from Datatype or Namespace str_def
25272
@@ -290,6 +110,26 @@ def str_type(long_name, single):
290110
return Namespace.add(long_name)
291111
return Datatype.add(long_name)
292112

113+
def _add_namespace(config, namesp):
114+
'''create the child Namespace and the child Datatype of the parent namespace'''
115+
if namesp.name in config.sections():
116+
confname = config[namesp.name]
117+
if 'namespace' in confname:
118+
for nspname in json.loads(confname['namespace']):
119+
nsp = Namespace(nspname, namesp, force=True)
120+
_add_namespace(config, nsp)
121+
if 'type' in confname:
122+
for typ in json.loads(confname['type']):
123+
Datatype(typ, namesp, force=True)
124+
125+
def _join_type(namesp, str_typ):
126+
'''join Namespace string and Datatype or Namespace string'''
127+
namesp_split = namesp.split('.')[:-1]
128+
for name in str_typ.split('.'):
129+
if not name in namesp_split:
130+
namesp_split.append(name)
131+
return '.'.join(namesp_split)
132+
293133

294134
class Datatype(NtvUtil):
295135
''' type of NTV entities.

‎json_ntv/ntv.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
11
# -*- coding: utf-8 -*-
22
"""
3-
Created on Feb 27 2023
4-
53
@author: Philippe@loco-labs.io
64
75
The `ntv` module is part of the `NTV.json_ntv` package ([specification document](

‎json_ntv/ntv_util.py

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,26 @@
77
The `ntv_util` module is part of the `NTV.json_ntv` package ([specification document](
88
https://github.com/loco-philippe/NTV/blob/main/documentation/JSON-NTV-standard.pdf)).
99
10-
It contains the classes `NtvConnector`, `NtvTree`, `NtvJsonEncoder` and `NtvError`
11-
for NTV entities.
10+
It contains the classes `NtvUtil`, `NtvConnector`, `NtvTree`, `NtvJsonEncoder`
11+
and `NtvError` for NTV entities.
1212
"""
1313
from abc import ABC, abstractmethod
1414
import datetime
1515
import json
1616

1717
class NtvUtil:
18-
''' The NtvUtil class includes static methods used by several NTV classes '''
18+
''' The NtvUtil class includes static methods used by several NTV classes.
19+
NtvUtil is the parent class of `Datatype`, `Namespace`, `Ntv`.
1920
21+
*class variables :*
22+
- **_namespaces_** : list of Namespace defined
23+
- **_types_** : list of Datatype defined
24+
25+
*static methods :*
26+
- `from_obj_name`
27+
- `decode_ntv_tab`
28+
29+
'''
2030
_namespaces_ = {}
2131
_types_ = {}
2232

@@ -513,8 +523,6 @@ def inner_nodes(self):
513523
if node.__class__.__name__ == 'NtvList']
514524

515525

516-
517-
518526
class NtvJsonEncoder(json.JSONEncoder):
519527
"""json encoder for Ntv data"""
520528

0 commit comments

Comments
 (0)