Skip to content

Commit

Permalink
search content pages
Browse files Browse the repository at this point in the history
  • Loading branch information
NilsOveTen committed Jul 17, 2024
1 parent e76a1cc commit 2ffbc5b
Show file tree
Hide file tree
Showing 7 changed files with 121 additions and 66 deletions.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Filter
title: LLM
weight: 2
---

TBD
TODO
47 changes: 47 additions & 0 deletions content/Felles datakatalog/Search/Search/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: Search
weight: 1
---

The service responsible for handling searches is [fdk-search-service](https://github.com/Informasjonsforvaltning/fdk-search-service), which is described by [this OpenAPI specification](https://raw.githubusercontent.com/Informasjonsforvaltning/fdk-search-service/main/openapi.yaml).

Production endpoint: <https://search.api.fellesdatakatalog.digdir.no/search>
Demo endpoint: <https://search.api.demo.fellesdatakatalog.digdir.no/search>
Staging endpoint: <https://search.api.staging.fellesdatakatalog.digdir.no/search>

Simple example using the staging endpoint:
```Shell
curl -X POST 'https://search.api.staging.fellesdatakatalog.digdir.no/search' -H 'Content-Type: application/json' -d '{"query":"test"}'
```

### Boosting

Hits from some fields will be prioritized over others, i.e. a matching hit from the title field will be prioritized over a hit from the description field

### Pagination

All search results will be paginated, it is possible to manipulate the size and page number with the pagination field in the search body.

Example using the pagination field:
```Shell
curl -X POST 'https://search.api.staging.fellesdatakatalog.digdir.no/search' -H 'Content-Type: application/json' -d '{"query":"test","pagination":{"size":1,"page":5}}'
```

### Filtering

It's possible to filter the search result, see SearchFilters in the OpenAPI specification for a list of all possible filters.

Example using the data theme filter:
```Shell
curl -X POST 'https://search.api.staging.fellesdatakatalog.digdir.no/search' -H 'Content-Type: application/json' -d '{"query":"test","filters":{"dataTheme":{"value":["ENVI"]}}}'
```

Example using the open data filter:
```Shell
curl -X POST 'https://search.api.staging.fellesdatakatalog.digdir.no/search' -H 'Content-Type: application/json' -d '{"query":"test","filters":{"openData":{"value":true}}}'
```

Example using the formats filter:
```Shell
curl -X POST 'https://search.api.staging.fellesdatakatalog.digdir.no/search' -H 'Content-Type: application/json' -d '{"query":"test","filters":{"formats":{"value":["MEDIA_TYPE application/json"]}}}'
```
54 changes: 54 additions & 0 deletions content/Felles datakatalog/Search/Sparql/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
title: SPARQL
weight: 3
---

[Read more about SPARQL here](https://www.w3.org/TR/sparql11-overview/).

Production endpoint: <https://sparql.fellesdatakatalog.digdir.no>
Demo endpoint: <https://sparql.demo.fellesdatakatalog.digdir.no>
Staging endpoint: <https://sparql.staging.fellesdatakatalog.digdir.no>

Simple example using the staging endpoint:
```Shell
curl -X POST 'https://sparql.staging.fellesdatakatalog.digdir.no/?query=SELECT%20%2A%20WHERE%20%7B%20?sub%20?pred%20?obj%20.%20%7D%20LIMIT%201'
```

Data.norge.no has a simple GUI for SPARQL queries:
- production <https://data.norge.no/sparql>
- demo <https://demo.fellesdatakatalog.digdir.no/sparql>
- staging <https://staging.fellesdatakatalog.digdir.no/sparql>

### Query examples

List all properties and objects where the subject is this dataset <https://staging.fellesdatakatalog.digdir.no/datasets/04edc67b-046c-37a8-9822-29f03d2f1e80>:

```shell
PREFIX dcat: <http://www.w3.org/ns/dcat#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX dct: <http://purl.org/dc/terms/>

SELECT ?property ?object

WHERE {​​​​​​
?dataset a dcat:Dataset .
?record foaf:primaryTopic ?dataset .
?record a dcat:CatalogRecord .
?record dct:identifier "04edc67b-046c-37a8-9822-29f03d2f1e80" .
?dataset ?property ?object .
}​​​​​​
```

List all dataset titles:

```shell
PREFIX dcat: <http://www.w3.org/ns/dcat#>
PREFIX dct: <http://purl.org/dc/terms/>

SELECT ?title

WHERE {​​​​​​​​​​​​​​
?dataset a dcat:Dataset .
?dataset dct:title ?title .
}​​​​​​​​​​​​​​​​​​​​​
```
18 changes: 18 additions & 0 deletions content/Felles datakatalog/Search/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
title: Search
weight: 8
---

There are 3 different ways to search the data in Data.norge.no, each is based on a different technology.

### Search

The main search service in Data.norge.no is based on ElasticSearch. This service only searches the text in a few fields, but has filters and other advanced functionality that helps users navigate.

### LLM

The LLM search service uses a 'Large Language Model' (LLM) that is built from the same data used in the main search, but will accept naturally worded queries where the other search demands exact wording of titles or keywords.

### SPARQL

The SPARQL search service is based on the RDF query language SPARQL, and is our most advanced and powerful search. A user will have to create queries that follows the correct syntax, but will be able to search all harvested data points, not just pre-selected fields.
38 changes: 0 additions & 38 deletions content/Felles datakatalog/Søk_og_filter/Sparql/_index.md

This file was deleted.

12 changes: 0 additions & 12 deletions content/Felles datakatalog/Søk_og_filter/Søk/_index.md

This file was deleted.

14 changes: 0 additions & 14 deletions content/Felles datakatalog/Søk_og_filter/_index.md

This file was deleted.

0 comments on commit 2ffbc5b

Please sign in to comment.