Redis Invokers

This package provides interactions with Redis. To date, only the retrieval of values from a Redis store is available.

Configuration

The Redis invoker expects a meta.yaml file that contains the following parameters:

connection: details around the connection to the Redis server

deduplicate: a boolean to indicate if keys should be deduplicated

Connection details are as follows:

host: the host URI to connect to. Can be overridden by environment variable REDIS_HOST.

port: the port to connect to. Can be overridden by the environment variable REDIS_PORT.

db: the number for the database to connect to. Overridden by environment variable REDIS_DB.

password: the password to use. Overridden by environment variable REDIS_PASSWORD.

backoff_max_time: the maximum time to wait between retries, in seconds. Default to 61 seconds. Can be overridden by environment variable REDIS_BACKOFF_MAX_TIME.

backoff_max_tries: the maximum number of retries to make. Defaults to 15 times. Can be overridden by environment variable REDIS_BACKOFF_MAX_TRIES.

Example

The following meta.yaml would define a RedisRetrieveInvoker. This invoker would connect to a Redis instance on port 6379 on the server called my_redis_host and retrieve values from database number 7. The invoker would deduplicate keys before retrieval.

invoker:
  type: genie_flow_invoker.invoker.redis.RedisRetrieveInvoker
  connection:
    host: my_redis_host
    port: 6379
    db: 7
    password: AapNootMies123
  deduplicate: True

RedisRetrieveInvoker

This invoker can retrieve the value for a key or a list of keys. If provided with a JSON-encoded list of keys, the return value will also be a list of values.

JSON

If the content sent to the invoker is a string that is not parseable as JSON, the complete content is used as the key. If the content is parseable, it can either be a JSON-encoded string or a JSON-encoded list of strings.

The returned value or values will always be JSON encoded. Meaning that, when a simple string is stored as a value for a key, it will be quoted. On the returned value, an attempt is made to parse the content as JSON. Meaning that, if the stored value is a JSON string, the return value will be a well-structured JSON rather than a string.

Input

ThisIsMyKey

will be used verbatim. So a retrieval is done using the key ThisIsMyKey.

"ThisIsMyKey"

will be parsed as a string and the retrieval is done using the key ThisIsMyKey.

["Key1", "Key2"]

will be parsed as a list of keys and retrieval is done using the list of keys being ["Key1", "Key2"].

Output

If the stored value is This is a value, then the returned value will be "This is a value", making it a properly structured JSON.

If the stored value is {"aap": 10}, then the returned value will be {"aap": 10}. The retrieval of multiple of these objects would be:

[{"aap": 10}, {"noot": 10}]

Deduplication

When the flag deduplicate has been set in the meta.yaml, then duplicate keys are removed. Whilst removing, the order is maintained. So for multiple occurrences of a key, only the first one is retained:

["aap", "noot", "aap", "mies"]

will become:

["aap", "noot", "mies"]

Non-existent keys

When retrieval is done for a key that does not exist, an empty string value is returned for that key.