Skip to content

Commit

Permalink
better transparency handling for structure recognizer (#2115)
Browse files Browse the repository at this point in the history
Some recognizable structures have "transparent" cells.  However, the Aho-Corasick algorithm doesn't support "wildcards", which is essentially what transparent cells are, for the purpose of matching.  So, for a structure template that contains transparent cells to be recognized, the cells of the world must also have "empty" cells in the same place.

Despite this, we can accommodate certain instances of structure recognition when the transparent cells are "overlapped" (or "incurred upon") by some other entity.  We do this by "masking out" entities from the world that don't participate in the set of structures we're trying to match against.

This often works well, except when the set of structures to be recognized contains:
* one structure `S1` with transparent cells
* another structure `S2` with some additional type of entity `E` not participating in `S1`
* The entity `E` occupies a transparent cell of `S1`.

`E` can't be masked out since we're trying to simultaneously recognize structures of type `S2` in the same Aho-Corasick pass.  So a legitimate instance of `S1` will fail to be recognized.

## Workaround

We can make multiple Aho-Corasick passes: one pass for each distinct set of "masked entities".  Notably:
* If no structures have transparent cells, we don't need to mask at all
* If all of the structures that contain transparent cells have an identical set of participating entities, they can all be handled in the same pass
* **Not implemented:** A structure with no transparency (i.e. with an empty "mask set") can be handled in the same pass as a a transparent structure, if they have an identical participating entity set.

## Caveats

Note that this fundamental limitation of Aho-Corasick still exists for "incurring entities" of the **same type** as the entity participants in the structure to be recognized.

E.g., if we have a structure defined as:
```
 X
XX
```
then, depending on the order of entity placement, the following occurrence in the world will **not** be recognized:
```
XX
XX
```

## Testing
```
scripts/test/run-tests.sh --test-options '--pattern "structure-recognizer"'
```

## Also in this PR
* improved docs/logging
  • Loading branch information
kostmo authored Aug 16, 2024
1 parent 268876f commit d1791a1
Show file tree
Hide file tree
Showing 10 changed files with 305 additions and 73 deletions.
2 changes: 2 additions & 0 deletions data/scenarios/Testing/1575-structure-recognizer/00-ORDER.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,5 @@
1575-bounding-box-overlap.yaml
1644-rotated-recognition.yaml
1644-rotated-preplacement-recognition.yaml
2115-encroaching-upon-exterior-transparent-cells.yaml
2115-encroaching-upon-interior-transparent-cells.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,6 @@ description: |
Additionally, recognition of statically-placed
structures at scenario initialization is also
unaffected by interior entities.
However, any such "contaminating" entities
will prevent the recognition of a structure
when constructed by a robot.
creative: false
objectives:
- teaser: Replace rock
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
version: 1
name: Structure recognition - exterior transparency
description: |
Incursion of an entity of a foreign type
upon a "transparent" cell within the bounding box
of a recognizable structure shall not prevent
the structure from being recognized.
If the incurring entity is the *same* type as
a participating entity in that structure, however,
it will prevent recognition.
creative: false
objectives:
- teaser: Recognize structure
goal:
- |
`chevron`{=structure} structure should be recognized upon completion,
even with an extraneous entity within its bounds.
condition: |
def isRight = \x. case x (\_. false) (\_. true); end;
foundBox <- structure "chevron" 0;
return $ isRight foundBox;
robots:
- name: base
dir: east
devices:
- ADT calculator
- blueprint
- fast grabber
- logger
- treads
inventory:
- [1, board]
- name: judge
dir: east
system: true
display:
invisible: true
solution: |
move; move; move;
swap "board";
structures:
- name: chevron
recognize: [north]
structure:
palette:
'b': [stone, board]
mask: '.'
map: |
.b
bb
- name: stripe
recognize: [north]
structure:
palette:
't': [grass, tree]
'b': [grass, board]
map: |
btb
known: [board, mountain, tree]
world:
dsl: |
{blank}
palette:
'.': [grass, erase]
'B': [grass, erase, base]
'j': [grass, erase, judge]
't': [grass, tree]
'b': [grass, board]
upperleft: [-7, 3]
map: |
j.....
......
.B.ttt
...bb.
......
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
version: 1
name: Structure recognition - interior transparency
description: |
Incursion of an entity of a foreign type
upon a "transparent" cell within the bounding box
of a recognizable structure shall not prevent
the structure from being recognized.
If the incurring entity is the *same* type as
a participating entity in that structure, however,
it will prevent recognition.
creative: false
objectives:
- teaser: Recognize structure
goal:
- |
`pigpen`{=structure} structure should be recognized upon completion,
even with an extraneous entity within its bounds.
condition: |
def isRight = \x. case x (\_. false) (\_. true); end;
foundBox <- structure "pigpen" 0;
return $ isRight foundBox;
robots:
- name: base
dir: east
devices:
- ADT calculator
- blueprint
- fast grabber
- logger
- treads
inventory:
- [1, board]
- name: judge
dir: east
system: true
display:
invisible: true
solution: |
move; move; move; move;
swap "board";
structures:
- name: pigpen
recognize: [north]
structure:
palette:
'b': [stone, board]
mask: '.'
map: |
bbbb
b..b
b..b
bbbb
- name: obstruction
recognize: [north]
structure:
palette:
't': [grass, tree]
'b': [grass, board]
map: |
tttb
known: [board, mountain, tree]
world:
dsl: |
{blank}
palette:
'.': [grass, erase]
'B': [grass, erase, base]
'r': [grass, mountain]
'j': [grass, erase, judge]
'p':
structure:
name: pigpen
cell: [grass]
'b':
structure:
name: obstruction
cell: [grass]
upperleft: [-7, 3]
map: |
j.....
.p....
...r..
B..b..
......
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ module Swarm.Game.Scenario.Topography.Structure.Recognition.Log where

import Data.Aeson
import Data.Int (Int32)
import Data.List.NonEmpty (NonEmpty)
import GHC.Generics (Generic)
import Servant.Docs (ToSample)
import Servant.Docs qualified as SD
Expand All @@ -14,8 +15,10 @@ import Swarm.Game.Scenario.Topography.Structure.Recognition.Type
import Swarm.Game.Universe (Cosmic)
import Swarm.Language.Syntax.Direction (AbsoluteDir)

type StructureRowContent e = [Maybe e]
type WorldRowContent e = [Maybe e]
-- | Type aliases for documentation
type StructureRowContent e = SymbolSequence e

type WorldRowContent e = SymbolSequence e

data OrientedStructure = OrientedStructure
{ oName :: OriginalName
Expand All @@ -27,7 +30,8 @@ distillLabel :: StructureWithGrid b a -> OrientedStructure
distillLabel swg = OrientedStructure (getName $ originalDefinition swg) (rotatedTo swg)

data MatchingRowFrom = MatchingRowFrom
{ rowIdx :: Int32
{ topDownRowIdx :: Int32
-- ^ numbered from the top down
, structure :: OrientedStructure
}
deriving (Generic, ToJSON)
Expand All @@ -45,14 +49,23 @@ data HaystackContext e = HaystackContext

data FoundRowCandidate e = FoundRowCandidate
{ haystackContext :: HaystackContext e
, structureContent :: StructureRowContent e
, rowCandidates :: [MatchingRowFrom]
, soughtContent :: StructureRowContent e
, matchedCandidates :: [MatchingRowFrom]
}
deriving (Functor, Generic, ToJSON)

data EntityKeyedFinder e = EntityKeyedFinder
{ searchOffsets :: InspectionOffsets
, candidateStructureRows :: NonEmpty (StructureRowContent e)
, entityMask :: [e]
-- ^ NOTE: HashSet has no Functor instance,
-- so we represent this as a list here.
}
deriving (Functor, Generic, ToJSON)

data ParticipatingEntity e = ParticipatingEntity
{ entity :: e
, searchOffsets :: InspectionOffsets
, entityKeyedFinders :: NonEmpty (EntityKeyedFinder e)
}
deriving (Functor, Generic, ToJSON)

Expand All @@ -63,14 +76,22 @@ data IntactPlacementLog = IntactPlacementLog
}
deriving (Generic, ToJSON)

data VerticalSearch e = VerticalSearch
{ haystackVerticalExtents :: InspectionOffsets
-- ^ vertical offset of haystack relative to the found row
, soughtStructures :: [OrientedStructure]
, verticalHaystack :: [WorldRowContent e]
}
deriving (Functor, Generic, ToJSON)

data SearchLog e
= FoundParticipatingEntity (ParticipatingEntity e)
| StructureRemoved OriginalName
| FoundRowCandidates [FoundRowCandidate e]
| FoundCompleteStructureCandidates [OrientedStructure]
| -- | There may be multiple candidate structures that could be
-- completed by the element that was just placed. This lists all of them.
VerticalSearchSpans [(InspectionOffsets, [OrientedStructure])]
VerticalSearchSpans [VerticalSearch e]
| IntactStaticPlacement [IntactPlacementLog]
deriving (Functor, Generic)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import Data.HashMap.Strict qualified as HM
import Data.HashSet qualified as HS
import Data.Hashable (Hashable)
import Data.Int (Int32)
import Data.List.NonEmpty (NonEmpty)
import Data.List.NonEmpty qualified as NE
import Data.Maybe (catMaybes)
import Data.Semigroup (sconcat)
Expand All @@ -32,13 +33,13 @@ mkOffsets pos xs =
-- rows constitute a complete structure.
mkRowLookup ::
(Hashable a, Eq a) =>
NE.NonEmpty (StructureRow b a) ->
NonEmpty (StructureRow b a) ->
AutomatonInfo a (SymbolSequence a) (StructureWithGrid b a)
mkRowLookup neList =
AutomatonInfo participatingEnts bounds sm
AutomatonInfo participatingEnts bounds sm tuples
where
mkSmTuple = entityGrid &&& id
tuples = NE.toList $ NE.map (mkSmTuple . wholeStructure) neList
tuples = NE.map (mkSmTuple . wholeStructure) neList

-- All of the unique entities across all of the full candidate structures
participatingEnts =
Expand All @@ -50,7 +51,7 @@ mkRowLookup neList =
mkOffsets rwIdx g

bounds = sconcat $ NE.map deriveRowOffsets neList
sm = makeStateMachine tuples
sm = makeStateMachine $ NE.toList tuples

-- | Make the first-phase lookup map, keyed by 'Entity',
-- along with automatons whose key symbols are "Maybe Entity".
Expand All @@ -61,7 +62,7 @@ mkRowLookup neList =
mkEntityLookup ::
(Hashable a, Eq a) =>
[StructureWithGrid b a] ->
HM.HashMap a (AutomatonInfo a (AtomicKeySymbol a) (StructureSearcher b a))
HM.HashMap a (NonEmpty (AutomatonInfo a (AtomicKeySymbol a) (StructureSearcher b a)))
mkEntityLookup grids =
HM.map mkValues rowsByEntityParticipation
where
Expand All @@ -75,17 +76,26 @@ mkEntityLookup grids =
structureRowsNE = NE.map myRow singleRows
sm2D = mkRowLookup structureRowsNE

mkValues neList = AutomatonInfo participatingEnts bounds sm
mkValues neList =
NE.map (\(mask, tups) -> AutomatonInfo mask bounds sm tups) tuplesByEntMask
where
participatingEnts =
HS.fromList
(concatMap (catMaybes . fst) tuples)
-- If there are no transparent cells,
-- we don't need a mask.
getMaskSet row =
if Nothing `elem` row
then HS.fromList $ catMaybes row
else mempty

tuples = HM.toList $ HM.mapWithKey mkSmValue groupedByUniqueRow
tuplesByEntMask = binTuplesHMasListNE $ NE.map (getMaskSet . fst &&& id) tuplesNE

tuplesNE = NE.map (\(a, b) -> (a, mkSmValue a b)) groupedByUniqueRow

groupedByUniqueRow =
binTuplesHMasListNE $
NE.map (rowContent . myRow &&& id) neList

groupedByUniqueRow = binTuplesHM $ NE.toList $ NE.map (rowContent . myRow &&& id) neList
bounds = sconcat $ NE.map expandedOffsets neList
sm = makeStateMachine tuples
sm = makeStateMachine $ NE.toList tuplesNE

-- The values of this map are guaranteed to contain only one
-- entry per row of a given structure.
Expand All @@ -111,6 +121,7 @@ mkEntityLookup grids =
SingleRowEntityOccurrences r e occurrences $
sconcat $
NE.map deriveEntityOffsets occurrences

unconsolidated =
map swap $
catMaybes $
Expand All @@ -123,7 +134,19 @@ mkEntityLookup grids =
binTuplesHM ::
(Foldable t, Hashable a, Eq a) =>
t (a, b) ->
HM.HashMap a (NE.NonEmpty b)
HM.HashMap a (NonEmpty b)
binTuplesHM = foldr f mempty
where
f = uncurry (HM.insertWith (<>)) . fmap pure

-- | We know that if the input to the binning function
-- is a nonempty list, the output map must also have
-- at least one element.
-- Ideally we would use a NonEmptyMap to prove this,
-- but unfortunately such a variant does not exist for 'HashMap'.
-- So we just "force" the proof by using 'NE.fromList'.
binTuplesHMasListNE ::
(Hashable a, Eq a) =>
NonEmpty (a, b) ->
NonEmpty (a, NonEmpty b)
binTuplesHMasListNE = NE.fromList . HM.toList . binTuplesHM
Loading

0 comments on commit d1791a1

Please sign in to comment.