Skip to content

Latest commit

 

History

History
81 lines (57 loc) · 2.11 KB

DESIGN.md

File metadata and controls

81 lines (57 loc) · 2.11 KB
Raw

High-level architecture overview of DIST-CLANG distributed compiler project

There are three components: Client, Emitter, Absorber

Client

  • Works locally
  • Communicates with Emitter via unix socket

What it does:

  1. Parses command line arguments
  2. Fixes file paths
  3. Defines a version of desired CLANG
  4. Generates message to EMITTER via protobuf

So, the aim for this component is to parse command line options, normalize paths and send further.

Emitter

  • Works locally

  • Works as a daemon

  • Communicates with Client via unix socket

  • Communicates with Absorber via TCP/IP network

  • There are two kinds of local cache:

    • DIRECT: checks DEPS files generated by previous compilation
    • SIMPLE: checks preprocessed file content

  • Cache key is a hash of :

    • Compiler version
    • Command-line arguments
    • Source file

What it does:

  1. Checks if a source file is in DIRECT cache

    • If so, output data

  2. Preprocess the source file

  3. Checks if the preprocessed file is in SIMPLE cache

    • If so, output data

  4. There is no cache entry for the particular source file

  5. Adds entry for compilation to a local queue

  6. There two kinds of worker which process items from the queue:

    1. Local worker: compiles file locally

    2. Remote worker: generate protobuf message to Absorber via protobuf with:

      • Preprocessed source file
      • Desired compiler version
      • Particular set of compilation flags

So, the aim for this component is to check if there is a source file is local caches (DIRECT and SIMPLE). If no, send a message for a remote compilation or compile it locally.

Absorber

  • Works on remote HW
  • Works with preprocessed source files
  • Communicates with EMITTER via network

What it does:

  1. Receives a message from EMITTER
  2. Compiles the preprocessed source file from message using a desired compiler
  3. Sends result back

So, the aim for this component is to compile the source file and send an output object file back.