Skip to content
This repository has been archived by the owner on Dec 20, 2018. It is now read-only.

TODO: Transform tiles to a stream of samples #7

Open
vsimko opened this issue Nov 6, 2016 · 3 comments
Open

TODO: Transform tiles to a stream of samples #7

vsimko opened this issue Nov 6, 2016 · 3 comments
Assignees

Comments

@vsimko
Copy link
Member

vsimko commented Nov 6, 2016

No description provided.

@aklink
Copy link
Member

aklink commented Dec 1, 2016

Prepared an example method "MultibandTile2LabelPoint" in UtilsSVM

  • class id is missing (currently placeholder 0.0 = Unclassified)
  • selection of "area of interest" per class missing
  • not elegant way (stitch + parallelize afterwards = not memory efficient, workflow not parallelized)
  • spatial information is lost: converting 2(+1) dimensional raster tile to 1(+1) dimensional point samples (note dimension+1: bands are converted to featurevector)

@aklink
Copy link
Member

aklink commented Feb 16, 2017

Since "Raster tile to pixels and back" is working, now the Spark Streaming / Kafka queue integration is missing.

@aklink
Copy link
Member

aklink commented Jul 6, 2017

TilePixelingExample can be extended by kafka producer in spark:

// Use UtilsKafka & UtilsSVM to create Kafka Stream from samples
    val (brokers, topic) = ("localhost:9092","neu")
    UtilsKafka.initKafka(topic)(brokers)
    val messages : Array[String] = UtilsSVM.toKafkaString(samples, UtilsSVM.Delimiter(";")).toLocalIterator.toArray
    val nmsg : Int = messages.length
    logger debug s"messages $nmsg"
    for ( imsg <- 0 until nmsg ){
      val str = messages(imsg)
      UtilsKafka.send(str)
    }

testet with: kafka_2.11-0.10.1.1$ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic neu --from-beginning :

1.0;2109.0;1965.0;1857.0;2653.0;SpatialKey(69005,41753);36;144
1.0;1827.0;1733.0;1658.0;2315.0;SpatialKey(69005,41753);36;145
1.0;1727.0;1650.0;1587.0;2194.0;SpatialKey(69005,41753);36;146
1.0;1727.0;1650.0;1587.0;2194.0;SpatialKey(69005,41753);36;147
1.0;1727.0;1650.0;1587.0;2194.0;SpatialKey(69005,41753);36;148
1.0;1727.0;1650.0;1587.0;2194.0;SpatialKey(69005,41753);36;149
1.0;1727.0;1650.0;1587.0;2194.0;SpatialKey(69005,41753);36;150
1.0;1727.0;1650.0;1587.0;2194.0;SpatialKey(69005,41753);36;151
1.0;1727.0;1650.0;1587.0;2194.0;SpatialKey(69005,41753);36;152
1.0;1727.0;1650.0;1587.0;2194.0;SpatialKey(69005,41753);36;153
1.0;1727.0;1650.0;1587.0;2194.0;SpatialKey(69005,41753);36;154
1.0;1734.0;1647.0;1588.0;2189.0;SpatialKey(69005,41753);36;155
1.0;1740.0;1644.0;1590.0;2184.0;SpatialKey(69005,41753);36;156

but the "reverse" channel is still missing and metadata ist lost (e.g. zoomlevel, SRS, extent)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants