File URL paths

[meeki007]

File URL paths ---> https://nodejs.org/api/fs.html#fs_file_url_paths

face: {
    detector: { modelPath: 'file://models/faceboxes.json' }
}

I am a dependent of your module

humanLibrary = require('@vladmandic/human/dist/human.node.js').default;

User OS's and how people decide to install stuff, @vladmandic/human can end up at different paths
So i resolve for you JSON and find it So I can use the models

var humanModulePath = path.dirname(require.resolve("@vladmandic/human/package.json"));

I then get a nice path, value of humanModulePath

/home/meeki/node-red-contrib-human-recognition/node_modules/@vladmandic/human

I try pluggin this into your

detector: { modelPath: 'file:/' + correctedModulePath + 'models/faceboxes.json' }

and I get:

human is not ready Error: Path /home/meeki/home/meeki/node-red-contrib-human-recognition/node_modules/@vladmandic/humanmodels/faceboxes.json does not exist: loading failed"

I know I can write up some spaghetti code using .indexOf('/') to correct for it but there is also the issue that require.resolve will use what ever the OS uses for paths. and reading up on File URL paths tells me I'll have to do something like.

var forceLinuxPath = humanModulePath.replace(/\\/g, "/");

before I go coding around this are you willing to code it up so it works as a dependent package ?

also I may just make a copy of the models and make my own folder but then its duplicating a bunch of files and increasing the install size of the package.

Is my brain right ?
Am I just missing a simple solution to this ?

Thanks you,
Meeki

[meeki007]

oh I just noticed i missed a '/' ..... fixed and still an issue

detector: { modelPath: 'file:/' + correctedModulePath + '/models/faceboxes.json' }

human is not ready Error: Path /home/meeki/home/meeki/node-red-contrib-human-recognition/node_modules/@vladmandic/human/models/faceboxes.json does not exist: loading failed

[vladmandic]

instead of adding strings, better to use path.join so nodejs takes care of directory separators.

detector: { modelPath: 'file:/' + path.join(correctedModulePath, 'models/faceboxes.json') }

[meeki007]

My poor brain

If I use

file:///home/meeki/node-red-contrib-human-recognition/node_modules/@vladmandic/human/models/faceboxes.json

I get

TypeError: Only absolute URLs are supported

But i know its a good path. Lets change meeki to some false path like notmeeki

file:///home/notmeeki/node-red-contrib-human-recognition/node_modules/@vladmandic/human/models/faceboxes.json

I get

Error: Path /home/notmeeki/node-red-contrib-human-recognition/node_modules/@vladmandic/human/models/faceboxes.json does not exist: loading failed"

I don't understand :(

[vladmandic]

i've just tried with both absolute and relative file:// paths and it works in my nodejs test.
i assume it's about node-red definition of working environment, it sounds like nodejs doesn't know its starting folder - what is the output of process.env.PWD?

i've also just tried with something like:

config.body.modelPath = new URL('/posenet.json', 'file://models').toString();

and it also loaded model correctly (note that models/ is a relative folder to current working folder)

[meeki007]

Thank you! I never thought about just using the object like that.....!
so you pointed me in the right direction!

I got rid of the symbolic link to the project. Published it to npm and then installed it like a user would to get proper dir structure for testing

my package @ /home/meeki/.node-red/node_modules/node-red-contrib-human-recognition
human dependency installs @ /home/meeki/.node-red/node_modules/@vladmandic/human

  myConfig.face.detector.modelPath = new URL('/faceboxes.json', 'file://.node-red/node_modules/@vladmandic/human/models').toString();

output is

Error: Path /home/meeki/.node-red/faceboxes.json does not exist: loading failed

So I then thought .... I'll just throw the faceboxes.json and .bin in that dir and give it what it wants....well......

TypeError: Only absolute URLs are supported

output of process.env.PWD

/home/meeki

However output of ${__dirname}

/home/meeki/.node-red/node_modules/node-red-contrib-human-recognition

I've even copied the models dir to /home/meeki/.node-red/node_modules/node-red-contrib-human-recognition/models

Gerr im missing something...... I don't expect you to be my tech support. If you don't have time to respond I'll be fine.

Going to read.....ALLOT . Its kinda fun when you figure it out. I enjoy the challenge.
Feel free to point me in the right direction but not give out the answer if you know it.

Thanks M8,
meeki

[meeki007]

Id hate to make an HTTP end-point and point it to it to make it work. But that my be my next option.

[meeki007]

can you tell location/js file and line number that handles

detector: { modelPath: }

I'd like to run some tests :)

[vladmandic]

quick test program to load model (in the same manner that human does):

const tf = require('@tensorflow/tfjs-node');

async function main() {
  const modelPath = 'file://models/blazepalm/graph/blazepalm.json';
  console.log('model path:', modelPath);
  console.log('platform: browser', tf.env().getBool('IS_BROWSER'), 'node', tf.env().getBool('IS_NODE'));
  const model = await tf.loadGraphModel(modelPath);
  console.log('loaded using:', model.handler);
  console.log('bytes used:', tf.engine().state.numBytes);
  console.log('model signature:', model.signature);
  tf.dispose(model);
}

main();

$ node test.js
model path: file://models/blazepalm/graph/blazepalm.json
platform: browser false node true
loaded using: NodeFileSystem {
  path: '/home/vlado/dev/detector/models/blazepalm/graph/blazepalm.json'
}
bytes used: 7373620
model signature: {
  inputs: {
    'img:0': { name: 'img:0', dtype: 'DT_FLOAT', tensorShape: [Object] }
  },
  outputs: {
    'Identity_1:0': { name: 'Identity_1:0', dtype: 'DT_FLOAT', tensorShape: [Object] },
    'Identity:0': { name: 'Identity:0', dtype: 'DT_FLOAT', tensorShape: [Object] }
  }
}

actual load function is provided by @tensorflow/tfjs-converter/src/executor/graph_models.ts:loadGraphModel()
which internally uses findIOHandler() which then goes down code path of
const handlers = io.getLoadHandlers(path as string, this.loadOptions);

now, default io handler is platform-default for fetch function,
but it can be overriden in options with either a custom io handler or a custom fetch function or it gets auto-detected
so, for urls starting with file:// and running inside nodejs, io handler should resolve to nodeFileSystemRouter

in your case, it seems that handler stays as platform fetch and then fetch expectedly fails with
Only absolute URLs are supported

and to confirm the theory, that's the same error i get if i manually call tf.env().platform.fetch(modelPath);
correct io handler would not use fetch as all, it would go down the path of using fs.readFile

in case of node-red, it seems that it somehow fails to change io handler from default one to filesystem one which can be bacause tfjs thinks it's not running in node?
i've added print of what tfjs things the platform is.

[meeki007]

Got to goto work but these are my objects from the test:

{"MODEL_JSON_FILENAME":"model.json","WEIGHTS_BINARY_FILENAME":"weights.bin","MODEL_BINARY_FILENAME":"tensorflowjs.pb","path":"/home/meeki/.node-red/node_modules/@vladmandic/human/models/faceboxes.json"}
{"outputs":{"scores":{"name":"scores"},"boxes":{"name":"boxes"},"num_boxes":{"name":"num_boxes"}}}

platform: browserfalsenodetrue
bytes used:4024860

I'll clean it up code later tonight and make it output pretty :)

[vladmandic]

good!
what did you change to make it work?

and ignore this part: MODEL_JSON_FILENAME":"model.json","WEIGHTS_BINARY_FILENAME":"weights.bin","MODEL_BINARY_FILENAME":"tensorflowjs.pb"

those are just default values in TFJS NodeFileSystem handler.
its a cosmetic bug for TFJS that it prints default values instead of resolved ones.

[meeki007]

oh not good. I got that from running the : "quick test program to load model (in the same manner that human does):"
Changed to work in node-red. Quick n dirty but wanted to test it.

        ////////////////////////////////////////////////////////
          //const modelPath = 'file://.node-red/node_modules/@vladmandic/human/models/faceboxes.json';
          modelPath = 'file://.node-red/node_modules/@vladmandic/human/models/faceboxes.json';
          notify_user_errors('model path:' + modelPath);
          notify_user_errors('platform: browser' + tf.env().getBool('IS_BROWSER') + 'node' + tf.env().getBool('IS_NODE'));
          model = await tf.loadGraphModel(modelPath);
          notify_user_errors('loaded using:' + model.handler);
          notify_user_errors('bytes used:' + tf.engine().state.numBytes);
          notify_user_errors('model signature:' + model.signature);
          msg.modelhandler = model.handler;
          msg.modelsignature = model.signature;
          tf.dispose(model);
        //////////////////////////////////////////////////////
        send(msg);

[vladmandic]

ah, understood. let me know.

[meeki007]

Will do.
It plays nice with ts

Its upset with (mynode) ---> human ---> to ts chain and its return.

I'm going to start diggin at human I don't understand type script so this could take a while.

[vladmandic]

Not sure what "plays nice with ts" means?
Does the quick earlier test work in node-red?
If yes, but human doesn't, still same thing - need to explore which handler gets used to load models.

[meeki007]

Not sure what "plays nice with ts" means?
Typo should have been *tf for @tensorflow/tfjs-node

Does the quick earlier test work in node-red?
yes the test works in node-red

loading models handler.............
for now I've gone to basics. Writing a non node-red js
testing things like my use of promises in the non node-red.js that is depending on your node

So far its looking to be a node-red issue and that is not your responsibility. As the node-red free node that is installed using:

npm install ~/home/meeki/package/humantest

is working fine. loves my promises and spits out all the checks as good.

sorry if I've wasted your time.
I will keep you posted on progress though.

[meeki007]

so the file handler from node-red to direct @tensorflow/tfjs-node works
but not from node-red to @vladmandic/human/dist/human.node.js
is my guess so far. reading up on how to track the chain for that and see whats switching and when.

[meeki007]

FOUND IT!!!!

error msg sent me down the wrong rabbit hole!

If the config is not perfectly formatted object it sends:

TypeError: Only absolute URLs are supported

Example failure by re-naming Object key to bodies instead of body

const myConfig = {
      backend: 'tensorflow',
      console: true,
      videoOptimized: false,
      async: false,
      face: {
        detector: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/faceboxes.json' }, // cannot use blazeface in nodejs due to missing required kernel function in tfjs-node
        mesh: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/facemesh.json' },
        iris: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/iris.json' },
        age: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/age-ssrnet-imdb.json' },
        gender: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/gender-ssrnet-imdb.json' },
        emotion: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/emotion-large.json' },
      },
      bodys: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/posenet.json' },
      hand: {
        detector: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/handdetect.json' },
        skeleton: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/handskeleton.json' },
      },
    };

Output

TypeError: Only absolute URLs are supported

Or if you try to send it an object that is incomplete

face: {
    detector: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/faceboxes.json' }, // cannot use blazeface in nodejs due to missing required kernel function in tfjs-node
}

Output

TypeError: Only absolute URLs are supported

You have to give every object key a value.
Working example:

    const myConfig = {
      backend: 'tensorflow',
      console: true,
      videoOptimized: false,
      async: false,
      face: {
        detector: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/faceboxes.json' }, // cannot use blazeface in nodejs due to missing required kernel function in tfjs-node
        mesh: { enabled: false },
        iris: { enabled: false },
        age: { enabled: false },
        gender: { enabled: false },
        emotion: { enabled: false },
      },
      body: { enabled: false },
      hand: { enabled: false },
    };

Output:

"Human:0.20.9"
"Current folder:/home/meeki"
"Active Configuration:{"backend":"tensorflow","wasmPath":"../assets/","async":false,"profile":false,"deallocate":false,"scoped":false,"videoOptimized":false,"warmup":"face","filter":{"enabled":true,"width":0,"height":0,"return":true,"brightness":0,"contrast":0,"sharpness":0,"blur":0,"saturation":0,"hue":0,"negative":false,"se.............."
"TFJS Version:"2.8.6""
"TFJS Backend:"tensorflow""
"TFJS Flags:{"IS_BROWSER":false,"IS_NODE":true,"DEBUG":false,"PROD":true}"
"Loading models:"
"  Loaded:facemesh"
"  Loaded:posenet"
"  Loaded:handpose"
"  Loaded:iris"
"  Loaded:age"
"  Loaded:gender"
"  Loaded:emotion"
"Memory state:{"unreliable":true,"numTensors":350,"numDataBuffers":350,"numBytes":4024860}"
"Test Complete"

as you can see its now loaded :)

You outline it in your https://github.com/vladmandic/human/wiki/Configuration but I missed the part where you have to send the Object key somting.

[meeki007]

So you may be wondering how i figured this one out. I would love to say some tecnobable about how i did this and that.......nope.....I was playing around with the nodeJS NON-node-red version and decided to try just facial rec, no other options,like i did with the node-version.

Got the same output I was getting in the node-red version and went .........."What did I just do....What did I change.......How did i make this happen". Played around for 10min with code and put together what was happening.

Corrected the object in the node-red version and BINGO!

[vladmandic]

interesting!

how are you using object myConfig?
for example, if you override human.config such as

const human = new Human();
human.config = myConfig

then I'd expect it to fail if its incomplete as you just overwrote defaults, but if you pass it during construction or load such as

const human = new Human(myConfig);

then human should perform object merge where it only takes defined and valid parts of your configuration and superimposes it on default config:

if (userConfig) this.config = mergeDeep(this.config, userConfig);

// helper function: perform deep merge of multiple objects so it allows full inheriance with overrides
function mergeDeep(...objects) {
  const isObject = (obj) => obj && typeof obj === 'object';
  return objects.reduce((prev, obj) => {
    Object.keys(obj || {}).forEach((key) => {
      const pVal = prev[key];
      const oVal = obj[key];
      if (Array.isArray(pVal) && Array.isArray(oVal)) prev[key] = pVal.concat(...oVal);
      else if (isObject(pVal) && isObject(oVal)) prev[key] = mergeDeep(pVal, oVal);
      else prev[key] = oVal;
    });
    return prev;
  }, {});
}

[meeki007]

how are you using object myConfig?
only removed code above and below the code block below.
only found 2 results of myConfig in the entire JS file and they are in the section below.
used //************************// to mark key sections for you

//************************//
const humanLibrary = require('@vladmandic/human/dist/human.node.js').default; // or const Human = require('../dist/human.node-gpu.js').default;
//************************//

  ///////////////////////////////////
  //Global stuff used by entire node
  ///////////////////////////////////

  //blabla code here

  ////////////////////////
  // construct the node //
  ////////////////////////
  function humanrecognitionNode(config)
  {
    RED.nodes.createNode(this,config);

    let human = null;

    //************************//
    const myConfig = {
    //************************//
      backend: 'tensorflow',
      console: true,
      videoOptimized: false,
      async: false,
      face: {
        detector: { modelPath: 'file://.node-red/node_modules/node-red-contrib-human-recognition/models/faceboxes.json' }, // cannot use blazeface in nodejs due to missing required kernel function in tfjs-node
        mesh: { enabled: false },
        iris: { enabled: false },
        age: { enabled: false },
        gender: { enabled: false },
        emotion: { enabled: false },
      },
      body: { enabled: false },
      hand: { enabled: false },
    };

    //is tfjs_ready
    var tfjs_ready; //error check of tfjs_ready
    Promise.resolve
    (
      tf.ready()
    )
    .then( tfjs_ready = true )
    .catch(error =>
      {
      tfjs_ready = ("tfjs is not ready " + error),
      this.warn(tfjs_ready),
      this.status(
      {
        fill: 'red',
        shape: 'dot',
        text: "detected error"
      });
    });

    //is human_ready - pre-load models
    var human_ready; //error check of human_ready

    //************************//
    human = new humanLibrary(myConfig);
    //************************//
    this.warn('Human:' + human.version);
    this.warn('Current folder:' + process.env.PWD);
    this.warn('Active Configuration:' + JSON.stringify(human.config));
    this.warn('TFJS Version:' + JSON.stringify(tf.version_core));
    this.warn('TFJS Backend:' + JSON.stringify(tf.getBackend()));
    this.warn('TFJS Flags:' + JSON.stringify(tf.env().features));
    this.warn('Loading models:');

    Promise.resolve
    (
      //************************//
      human.load()
      //************************//
    )
    .then( human_ready = true )
    .catch(error =>
      {
      human_ready = ("human is not ready " + error),
      this.warn(human_ready),
      this.status(
      {
        fill: 'red',
        shape: 'dot',
        text: "detected error"
      });
    });

    for (const model of Object.keys(human.models)) this.warn('  Loaded:' + model);
    this.warn('Memory state:' + JSON.stringify(human.tf.engine().memory()));
    this.warn('Test Complete');

I don't think I overrode the config........

[meeki007]

NOTE: I would love to use async in the node-red constructor, but the class constructor function for node-red does not support async/await directly.
I can use async later when passing a triggered input

var node = this;
    this.on('input', async function(msg, send, done)

but if I want to load human so its ready for a user input (buffered image) I have to use promises in the constructor. .....there is a method for my madness.

else if I load human models in the user input (buffered image) section with async It would take the time to load models again and again for every image the user sends.......thats allot of overhead

[vladmandic]

but you ARE passing your config as object to human constructor which is correct:

    human = new humanLibrary(myConfig);

which means default config and your config should be merged and any possible invalid config entries in user config should not matter.

can you reproduce error using bad config as before and post output of:

    this.warn('Human:' + human.version);

so i can check what happened with the merge?

[meeki007]

you ask for - this.warn('Human:' + human.version); I think you want - this.warn('Active Configuration:' + JSON.stringify(human.config));

This is partly my fault. I limit the amount of char my debug can send per msg so I don't risk the chance of a lock up.
So when I got the output for - human.config - It is chopped off before I could notice the end.

Here is the complete output for - human.config - using bodys insted of body for myConfig just like the example a few threads above.

Active Configuration:{"backend":"tensorflow","wasmPath":"../assets/","async":false,"profile":false,"deallocate":false,"scoped":false,"videoOptimized":false,"warmup":"face","filter":{"enabled":true,"width":0,"height":0,"return":true,"brightness":0,"contrast":0,"sharpness":0,"blur":0,"saturation":0,"hue":0,"negative":false,"sepia":false,"vintage":false,"kodachrome":false,"technicolor":false,"polaroid":false,"pixelate":0},"gesture":{"enabled":true},"face":{"enabled":true,"detector":{"modelPath":"file://.node-red/node_modules/node-red-contrib-human-recognition/models/faceboxes.json","inputSize":256,"rotation":false,"maxFaces":10,"skipFrames":11,"minConfidence":0.2,"iouThreshold":0.2,"scoreThreshold":0.2},"mesh":{"enabled":false,"modelPath":"../models/facemesh.json","inputSize":192,"returnRawData":false},"iris":{"enabled":false,"modelPath":"../models/iris.json","inputSize":64},"age":{"enabled":false,"modelPath":"../models/age-ssrnet-imdb.json","inputSize":64,"skipFrames":31},"gender":{"enabled":false,"minConfidence":0.4,"modelPath":"../models/gender.json","inputSize":64,"skipFrames":41},"emotion":{"enabled":false,"inputSize":64,"minConfidence":0.2,"skipFrames":21,"modelPath":"../models/emotion-large.json"},"embedding":{"enabled":false,"inputSize":112,"modelPath":"../models/mobilefacenet.json"}},"body":{"enabled":true,"modelPath":"../models/posenet.json","inputSize":257,"maxDetections":10,"scoreThreshold":0.5,"nmsRadius":20,"outputStride":16,"modelType":"MobileNet"},"hand":{"enabled":false,"rotation":false,"inputSize":256,"skipFrames":12,"minConfidence":0.1,"iouThreshold":0.1,"scoreThreshold":0.5,"maxHands":1,"landmarks":true,"detector":{"modelPath":"../models/handdetect.json"},"skeleton":{"modelPath":"../models/handskeleton.json"}},"console":true,"bodys":{"enabled":false}}"

As you can see it adds your:

"body":{"enabled":true,"modelPath":"../models/posenet.json"..............

back in and then adds my incorrect object to the bottom:

"bodys":{"enabled":false}}"

I have been ignoring the output from your checks:

for (const model of Object.keys(human.models)) this.warn('  Loaded:' + model);

because it only checks for the key and not its value. So even though I may use:

iris: { enabled: false },

It will still give me an output:

"  Loaded:iris"

I will code up a check that looks for the object key and its value so if its false it will tell me its not loaded.

Then the last output I get is the failure of the Promise for human.load():

human is not ready TypeError: Only absolute URLs are supported

for the failure of:

"body":{"enabled":true,"modelPath":"../models/posenet.json"..............

I hope this gives you the information you need.

Oh the output of this.warn('Human:' + human.version);

Human:0.20.9

[vladmandic]

I have been ignoring the output from your checks:
because it only checks for the key and not its value. So even though I may use: iris: { enabled: false },
It will still give me an output: " Loaded:iris"

that sounds like a bug
yes, it checks for keys only, but key should not be set if model is not loaded
will check

[vladmandic]

changed logic to

  const loaded = Object.keys(human.models).filter((a) => human.models[a]);
  logger.info('Loaded:', loaded);

however, note that the output may not match 'loaded' messages as blazeface and facemesh are merged during load into single model face and simmilarly handdetect and handskeleton are merged into handpose. equally, emotion is always emotion regardless which variation of emotion*.json you're loading - it means emotion model is loaded. same for gender and age.

basically, this shows which models are constructed inside human.

[meeki007]

nice! I'll give it a try late late tonight.

I'm marking this as answered. If a user sends the wrong object and it defaults to stock path then that is good enough.
If someone is having an issue with TypeError: Only absolute URLs are supported you / I will ask "what is the value of your human.config after await/promise of human.load()

I've just now added a watch to issues and discussions of this repository. If I can help answer something I will.

[vladmandic]

Sounds good!