Store each watch config in its own directory, saves rewriting the whole json file each time

Store each watch config in its own directory, saves rewriting the whole json file each time #6406