How to ignore importing entire extratags column in placex table?

[azmicirit]

Is your feature request related to a problem? Please describe.
I'm not able to ignore importing extratags column in placex table?

Describe the solution you'd like
I updated my .env file to point my new import-extratags.lua script like this:
NOMINATIM_IMPORT_STYLE=/srv/nominatim/nominatim-project/styles/import-extratags.lua

Created a new file and tried some lua scripts to ignore all tags and make my extratags columm completely empty. This is one of them:

local flex = require('flex-base')

function set_main_tags(tags, extratags, geometry)
  -- Ensure tags are processed; ignore extratags
  if not tags or next(tags) == nil then
      return nil  -- Skip objects with no tags
  end
  return tags
end
function set_prefilters(filters, geometry)
  -- Use default filters to ensure valid data is processed
  filters.tagfilter = function(tags, geometry)
      -- Example: Only process objects with at least one main tag
      if tags and next(tags) ~= nil then
          return true  -- Process this object
      end
      return false  -- Exclude objects with no tags
  end
  return filters
end

function set_name_tags(filters, geometry)
    -- Do not add any prefilters for extratags
    return filters
end

function set_address_tags(filters, geometry)
    -- Do not add any prefilters for extratags
    return filters
end

function set_unused_handling(filters, geometry)
    -- Do not add any prefilters for extratags
    return filters
end

return {
    set_main_tags = set_main_tags,
    set_prefilters = set_prefilters,
    set_name_tags = set_name_tags,
    set_address_tags = set_address_tags,
    set_unused_handling = set_unused_handling
}

This one returns this error: FATAL: No data imported by osm2pgsql.

Describe alternatives you've considered

local flex = require('flex-base')

flex.set_main_tags{}

flex.set_prefilters{delete_keys = {'*'}
                   }

flex.set_name_tags{main = {'name', 'name:*',
                          'int_name', 'int_name:*',
                          'nat_name', 'nat_name:*',
                          'reg_name', 'reg_name:*',
                          'loc_name', 'loc_name:*',
                          'old_name', 'old_name:*',
                          'alt_name', 'alt_name:*', 'alt_name_*',
                          'official_name', 'official_name:*',
                          'place_name', 'place_name:*',
                          'short_name', 'short_name:*', 'brand'},
                   extra = {'ref', 'int_ref', 'nat_ref', 'reg_ref',
                            'loc_ref', 'old_ref',
                            'iata', 'icao', 'pcode', 'pcode:*', 'ISO3166-2'},
                   house = {'addr:housename'}
                  }

flex.set_address_tags{main = {'addr:housenumber',
                              'addr:conscriptionnumber',
                              'addr:streetnumber'},
                      extra = {'addr:*', 'is_in:*', 'tiger:county'},
                      postcode = {'postal_code', 'postcode', 'addr:postcode',
                                  'tiger:zip_left', 'tiger:zip_right'},
                      country = {'country_code', 'ISO3166-1',
                                 'addr:country_code', 'is_in:country_code',
                                 'addr:country', 'is_in:country'},
                      interpolation = {'addr:interpolation'}
                     }


flex.set_unused_handling{delete_keys = {'tiger:*'}}

return flex

This one returns successful importing process but there are many records in extratags column in placex

Additional context
I tried to update all columns NULL in extratags column in postgresql 16 however my database size is getting bigger because of vacuum problem in this case which is much more trouble.

I have limited space in server and I don't need entire extratags column in my database. Please help me to ignore importing entire extratags columns in placex and place tables.

[mtmail]

Using import-full.lua should import only few extra tags. Some are required, for example the wikipedia tags for importance scoring. Nominatim will write other data to the column as well, for example the key linked_place for administrative areas.

What is the main goal? Saving disc space for an existing database?

A quick fix would be to delete any tables related to data updates (https://nominatim.org/release-docs/latest/admin/Import/#dropping-data-required-for-dynamic-updates).

If you have enough disc space you can try recreating the database indexes. https://www.postgresql.org/docs/current/sql-reindex.html That needs a bit of downtime.

[azmicirit]

@mtmail thank you for your response. Main goal is to use less disk space before importing pbf to postgredb via nominatim import command.

Actually, I've found a solution for it, however I couldn't achieve to overwrite a style import-extratags.lua so I had to edit it in settings/flex-base.lua in python lib (in nominatim-venv) directly

Path: /srv/nominatim/nominatim-venv/lib/python3.12/site-packages/nominatim_db/resources/settings/flex-base.lua

242     place_table:insert{
243         class = k,
244         type = v,
245         admin_level = self.admin_level,
246         name = next(self.names) and self.names,
247         address = next(self.address) and self.address,
248         --extratags = next(self.extratags) and self.extratags,
249         extratags = nil,
250         geometry = self.geometry
251     }

It helped me to ignore importing almost all data in extratags than "linked_place"=>"" key-values in placex table.
DB Size has been decreased from 147GB to 143GB as total.

I think 143GB is still big for only 2 countries (UK and Germany). I'm looking for less compact data set in nominatim database.

Environment:
Nominatim 4.5.0
Ubuntu 24.0.4 LTS
Python Frontend Virtualized
United-kingdom and Germany import from geofabrik.de.

[lonvia]

You really should listen to @mtmail and use the import-full.lua style. It will do exactly what you want. Your solution will cause the quality of search to degrade significantly. The next version of the Nominatim documentation will include a big warning that some tags are required in the extratags column and give you the right tools to include only those. For the current version, please refer to the pre-defined styles to ensure that the tables are filled optimally.

If you want to save space, extratags isn't really the place to start. Look into what main features are imported and throw away those you don't need. Things like park benches, for example. Another way would be to restrict what name tags are imported and stick to the languages you need. That could reduce the size of the search tables quite a bit.

[azmicirit]

@lonvia Thank you for reply. I will follow your guide to save some spaces in database. I will check the import-full.lua style and do some tests on small regions first.
On other hand, I will try to analyze the requirements and check tags deeply.