Port to ocrd core version 3.0.0 #5
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -31,16 +31,23 @@ | |
| pil2array, array2pil | ||
| ) | ||
|
|
||
| class OcropyClip(Processor): | ||
|
|
||
| def __init__(self, *args, **kwargs): | ||
| self.logger = getLogger('processor.OcropyClip') | ||
| self.ocrd_tool = get_ocrd_tool() | ||
| kwargs['ocrd_tool'] = self.ocrd_tool['tools'][self.executable] | ||
| kwargs['version'] = self.ocrd_tool['version'] | ||
| super(OcropyClip, self).__init__(*args, **kwargs) | ||
|
|
||
| @property | ||
| def executable(self): | ||
| return 'ocrd-cis-ocropy-clip' | ||
|
|
||
| def setup(self): | ||
| assert_file_grp_cardinality(self.input_file_grp, 1) | ||
| assert_file_grp_cardinality(self.output_file_grp, 1) | ||
|
|
||
| def process(self): | ||
| """Clip text regions / lines of the workspace at intersections with neighbours. | ||
|
|
||
|
|
@@ -76,13 +83,12 @@ def process(self): | |
| # too. However, region-level clipping _must_ be run before region-level | ||
| # deskewing, because that would make segments incomensurable with their | ||
| # neighbours. | ||
| level = self.parameter['level-of-operation'] | ||
| assert_file_grp_cardinality(self.input_file_grp, 1) | ||
| assert_file_grp_cardinality(self.output_file_grp, 1) | ||
|
|
||
| for (n, input_file) in enumerate(self.input_files): | ||
|
There was a problem hiding this comment. Shouldn't we migrate to There was a problem hiding this comment. Yes, everywhere. For now only binarize is migrated. I am just lagging with the migration since I have no working tests locally. The server from where the resources are downloaded is not available anymore. Hence, I try to adapt only things I understand and I am sure are the right things to do. There was a problem hiding this comment. Oops! Strange that Github pull the older release of our GT. Anyway, with OCR-D/gt_structure_text#2 out of the way it should now suffice to change the base URL in to There was a problem hiding this comment. And again, thanks for being so thorough! There was a problem hiding this comment. That still did not help. (venv38-core) mm@MM-Notebook:~/repos/ocrd_cis$ make test
bash tests/run_add_zip_test.bash > /dev/null 2>&1
make: *** [Makefile:25: tests/run_add_zip_test.bash] Error 1To get more detailed errors, I did: (venv38-core) mm@MM-Notebook:~/repos/ocrd_cis$ bash tests/run_add_zip_test.bash
--2024-08-14 14:47:30-- https://github.com/OCR-D/gt_structure_text/releases/tag/v1.5.0//blumenbach_anatomie_1805.ocrd.zip
Resolving github.com (github.com)... 140.82.121.3
Connecting to github.com (github.com)|140.82.121.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘/home/mm/repos/ocrd_cis/download/blumenbach_anatomie_1805.ocrd.zip’
blumenbach_anatomie_1805.ocrd.zip [ <=> ] 172,14K --.-KB/s in 0,08s
2024-08-14 14:47:30 (2,16 MB/s) - ‘/home/mm/repos/ocrd_cis/download/blumenbach_anatomie_1805.ocrd.zip’ saved [176273]
14:47:31.446 INFO ocrd.workspace_bagger - Spilling /home/mm/repos/ocrd_cis/download/blumenbach_anatomie_1805.ocrd.zip to /tmp/tmp.QcrQegkntf/blumenbach_anatomie_1805
Traceback (most recent call last):
File "/home/mm/venv38-core/bin/ocrd", line 8, in <module>
...
File "/home/mm/repos/core/build/__editable__.ocrd-2.67.2-py3-none-any/ocrd_utils/os.py", line 74, in unzip_file_to_dir
z = ZipFile(path_to_zip, 'r')
File "/usr/lib/python3.8/zipfile.py", line 1271, in __init__
self._RealGetContents()
File "/usr/lib/python3.8/zipfile.py", line 1338, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip fileSeems the download fails for some reason since the size of the zip is much smaller. I have manually downloaded the zip and placed it where it is expected. Hooray, the tests started passing, but then: (venv38-core) mm@MM-Notebook:~/repos/ocrd_cis$ make test
bash tests/run_add_zip_test.bash > /dev/null 2>&1
bash tests/run_alignment_test.bash > /dev/null 2>&1
make: *** [Makefile:25: tests/run_alignment_test.bash] Error 1There was a problem hiding this comment.
Yes, I have already done that. There was another other missing import - Levenshtein. After I have manually installed that, I am stuck again on the same error. (venv38-core-v3) mm@MM-Notebook:~/repos/ocrd_cis$ python --version
Python 3.8.16
(venv38-core-v3) mm@MM-Notebook:~/repos/ocrd_cis$ pip freeze
...
Levenshtein==0.25.1
...
numpy==1.24.4
-e git+ssh://git@github.com/bertsky/core.git@228272b6a4ee94795e8266af4182eacae38e713c#egg=ocrd
ocrd-cis @ file:///home/mm/repos/ocrd_cis
...
(venv38-core-v3) mm@MM-Notebook:~/repos/ocrd_cis$ ocrd --version
ocrd, version 3.0.0a1
(venv38-core-v3) mm@MM-Notebook:~/repos/ocrd_cis$ which ocrd
/home/mm/venv38-core-v3/bin/ocrdThere was a problem hiding this comment. It is strange because all other tests pass normally although invoking the same method and are supposed to fail. o.0 There was a problem hiding this comment. I also noticed missing Levenshtein and also broken There was a problem hiding this comment. Just do the broken There was a problem hiding this comment. Oops! Indeed, I forgot to update the non-ocropy processors in that regard. |
||
| self.logger.info("INPUT FILE %i / %s", n, input_file.pageId or input_file.ID) | ||
| file_id = make_file_id(input_file, self.output_file_grp) | ||
|
|
||
| pcgts = page_from_file(self.workspace.download_file(input_file)) | ||
|
|
@@ -98,7 +104,7 @@ def process(self): | |
| dpi = page_image_info.resolution | ||
| if page_image_info.resolutionUnit == 'cm': | ||
| dpi *= 2.54 | ||
| self.logger.info('Page "%s" uses %f DPI', page_id, dpi) | ||
| zoom = 300.0/dpi | ||
| else: | ||
| zoom = 1 | ||
|
|
@@ -120,7 +126,7 @@ def process(self): | |
| page.get_TableRegion() + | ||
| page.get_UnknownRegion()) | ||
| if not num_texts: | ||
| self.logger.warning('Page "%s" contains no text regions', page_id) | ||
| background = ImageStat.Stat(page_image) | ||
| # workaround for Pillow#4925 | ||
| if len(background.bands) > 1: | ||
|
|
@@ -151,7 +157,7 @@ def process(self): | |
| if level == 'region': | ||
| if region.get_AlternativeImage(): | ||
| # FIXME: This should probably be an exception (bad workflow configuration). | ||
| self.logger.warning('Page "%s" region "%s" already contains image data: skipping', | ||
| page_id, region.id) | ||
| continue | ||
| shape = prep(shapes[i]) | ||
|
|
@@ -169,7 +175,7 @@ def process(self): | |
| # level == 'line': | ||
| lines = region.get_TextLine() | ||
| if not lines: | ||
| self.logger.warning('Page "%s" region "%s" contains no text lines', page_id, region.id) | ||
| continue | ||
| region_image, region_coords = self.workspace.image_from_segment( | ||
| region, page_image, page_coords, feature_selector='binarized') | ||
|
|
@@ -187,7 +193,7 @@ def process(self): | |
| for j, line in enumerate(lines): | ||
| if line.get_AlternativeImage(): | ||
| # FIXME: This should probably be an exception (bad workflow configuration). | ||
| self.logger.warning('Page "%s" region "%s" line "%s" already contains image data: skipping', | ||
| page_id, region.id, line.id) | ||
| continue | ||
| shape = prep(shapes[j]) | ||
|
|
@@ -212,13 +218,12 @@ def process(self): | |
| local_filename=file_path, | ||
| mimetype=MIMETYPE_PAGE, | ||
| content=to_xml(pcgts)) | ||
| self.logger.info('created file ID: %s, file_grp: %s, path: %s', | ||
| file_id, self.output_file_grp, out.local_filename) | ||
|
|
||
| def process_segment(self, segment, segment_mask, segment_polygon, neighbours, | ||
| background_image, parent_image, parent_coords, parent_bin, | ||
| page_id, file_id): | ||
| # initialize AlternativeImage@comments classes from parent, except | ||
| # for those operations that can apply on multiple hierarchy levels: | ||
| features = ','.join( | ||
|
|
@@ -230,7 +235,7 @@ def process_segment(self, segment, segment_mask, segment_polygon, neighbours, | |
| segment_bbox = bbox_from_polygon(segment_polygon) | ||
| for neighbour, neighbour_mask in neighbours: | ||
| if not np.any(segment_mask > neighbour_mask): | ||
| self.logger.info('Ignoring enclosing neighbour "%s" of segment "%s" on page "%s"', | ||
| neighbour.id, segment.id, page_id) | ||
| continue | ||
| # find connected components that (only) belong to the neighbour: | ||
|
|
@@ -240,7 +245,7 @@ def process_segment(self, segment, segment_mask, segment_polygon, neighbours, | |
| num_foreground = np.count_nonzero(segment_mask * parent_bin) | ||
| if not num_intruders: | ||
| continue | ||
| self.logger.debug('segment "%s" vs neighbour "%s": suppressing %d of %d pixels on page "%s"', | ||
| segment.id, neighbour.id, num_intruders, num_foreground, page_id) | ||
| # suppress in segment_mask so these intruders can stay in the neighbours | ||
| # (are not removed from both sides) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.