Talk about the introduction and use of gin library of python

1. Introduction

Because many machine learning experiments need to set tedious parameters, some parameters are one in many experiments. In order to set parameters conveniently, Gin library appeared. It allows functions or classes to be annotated as @ gin. configurable, which makes it possible to set their parameters with a simple configuration file using a clear and powerful syntax. This method reduces configuration maintenance, and makes the experimental configuration transparent and easy to repeat.

Simply understand, gin is like a class that encapsulates parameter configuration. Using this class will make a lot of parameter configuration simple and clear

Installation

pip install gin-config

2.@gin.configurable

Any function and class can use the @ gin. configurable decorator

@gin.configurable
def my_network(images, num_outputs, num_layers=3, weight_decay=1e-4):
  ...

The @ gin.configurable decorator does the following three things:

Declare a class or function as something configurable It determines which parameters of a function or class constructor are configurable (all its parameters by default) Encapsulates a class or function, intercepts calls, and provides configurable parameters of the function with values from the parameter setting global registry that are not specified when the class or function is declared

In order to determine which parameters are configurable, @ gin. configurable will use allowlist and denylist parameters to declare which are configurable and which are not. We usually use one. By default, those not specified by allowlist are not configurable, and vice versa.

@gin.configurable('supernet', denylist=['images'])
def my_network(images, num_outputs, num_layers=3, weight_decay=1e-4):
  ...

Where supernet is the configuration name we specified.

3. Assignment

We use the following two formats to assign values to parameters:

gin.bind_parameter('configurable_name.parameter_name', value)
configurable_name.parameter_name = value

Specific examples are as follows:

gin.bind_parameter('supernet.num_layers', 5)
gin.bind_parameter('supernet.weight_decay', 1e-3)

supernet.num_layers = 5
supernet.weight_decay = 1e-3

4. Value

We can use gin.query_parameter To get the value, the specific example is as follows

num_layers = gin.query_parameter('supernet.num_layers')
weight_decay = gin.query_parameter('supernet.weight_decay')

5. Configure the reference file

Suppose we have the following code:

@gin.configurable
class DNN(object):
  def __init__(self, num_units=(1024, 1024)):
    ...
  def __call__(inputs, num_outputs):
    ...

@gin.configurable(denylist=['data'])
def train_model(network_fn, data, learning_rate, optimizer):
  ...

We can configure the parameters in the gin file:

train_model.network_fn = @DNN()  # An instance of DNN is passed.
train_model.optimizer = @MomentumOptimizer  # The class itself is passed.
train_model.learning_rate = 0.001

DNN.num_units = (2048, 2048, 2048)
MomentumOptimizer.momentum = 0.9

Two styles of configuration parameters are shown above. @DNN() And @MomentumOptimizer . For the former, instance parameters of the DNN class are called, and each parameter configuration varies with each instance of the DNN class. For the latter, the default parameters of class MomentumOptimizer will be called.

6. Use the gin file

We often use gin with flags1 under absl, such as the following

from absl import flags

flags.DEFINE_multi_string(
  'gin_file', None, 'List of paths to the config files.')
flags.DEFINE_multi_string(
  'gin_param', None, 'Newline separated list of Gin parameter bindings.')

FLAGS = flags.FLAGS

Then the main program main. py first parses the parameters:

gin.parse_config_files_and_bindings(FLAGS.gin_file, FLAGS.gin_param)

Assuming that our parameter file example. gin is in the current directory, when running, we enter it at the terminal python main.py --gin_file=example.gin

You can also change it to this in the code:

@gin.configurable
def my_network(images, num_outputs, num_layers=3, weight_decay=1e-4):
  ...

0

Then run it directly

6. Call another class or function

We can use the following code to call the parameters of other classes or functions, even if this class or function can be in other projects.

gin.external_configurable(tf.train.MomentumOptimizer)

7. Scope limits

When a configurable function is called multiple times during program execution, it may be necessary to provide different parameter bindings for each call. Gin provides a scoping mechanism to facilitate this point.
For example, suppose we want to implement an GAN, we must alternately train a generator and a discriminator. In Tensoflow, this is easiest to do with two optimizers, so we might have a function like this:

gin.external_configurable(tf.train.GradientDescentOptimizer)

@gin.configurable(allowlist=['generator_optimizer', 'discriminator_optimizer'])
def gan_trainer(
    generator_loss, generator_vars, generator_optimizer,
    discriminator_loss, discriminator_vars, discriminator_optimizer):
  # Construct the optimizers and minimize w.r.t. the correct variables.
  generator_train_op = generator_optimizer().minimize(
      generator_loss, generator_vars)
  discriminator_train_op = discriminator_optimizer().minimize(
      discriminator_loss, discriminator_vars)
  ...

How do we put generator_optimizer And discriminator_optimizer Are configured to @GradientDescentOptimizer But with different learning rates?
Here's an example of a mistake:

@gin.configurable
def my_network(images, num_outputs, num_layers=3, weight_decay=1e-4):
  ...

3

Gin provides a scoping mechanism to deal with this situation. Any configurable reference can be preceded by a scope name, separated from the configurable name by the/character. Similarly, you can apply a scope-specific binding by preceding the configurable name with a scope name.
The following is a demonstration of the right:

@gin.configurable
def my_network(images, num_outputs, num_layers=3, weight_decay=1e-4):
  ...

4

8. Marking the gin parameter

Gin allows you to indicate that certain parameters must be supplied in an Gin configuration. This can be achieved in two ways:
1. At the call location of the function
2. In the signature of the function

When 1 configurable is called, you can use the gin.REQUIRED Mark any arg or kwarg. Required objects:

@gin.configurable
def my_network(images, num_outputs, num_layers=3, weight_decay=1e-4):
  ...

5

The required parameters will be checked at call time. If no Gin bindings are provided for these parameters, an error is raised listing the missing parameter bindings and the configurable names that require them.
When defining configurable, you can use the gin.REQUIRED Marking parameters as required:

@gin.configurable
def my_network(images, num_outputs, num_layers=3, weight_decay=1e-4):
  ...

6

9. Import modules from an Gin file

import some.module.spec

10. Call another Gin file parameter in the Gin file

An Gin file can contain other Gin files, making it easier to split a configuration into separate components (for example, a "base" configuration that is included and modified by other derived configurations). Inclusion of another Gin file can be accomplished using the following syntax:

include 'path/to/another/file.gin'

11. Gin "macros"

Sometimes a value should be shared among multiple bindings. To achieve this point and avoid repeating this value many times (causing maintenance burden), Gin provides the following predefined configurable functions:

@gin.configurable
def my_network(images, num_outputs, num_layers=3, weight_decay=1e-4):
  ...

9

You can refer to the "macro" function (take the value by "()"). For example:

num_layers/macro.value = 10
network.num_layers = @num_layers/macro()

It can also be written like this

num_layers = 10
network.num_layers = %num_layers

12. Constants

gin.constant Function can be used to define constants, which can be accessed through the macro syntax mentioned above. For example, in Python:

gin.constant('THE_ANSWER', 42)

Then in the configuration file gin

meaning.of_life = %THE_ANSWER

Note that any Python object can be used as a constant value (including objects that cannot be represented as Gin literals). Values are stored in the Gin internal dictionary until the program terminates, thus avoiding the creation of constants with finite life-cycle values.
A disambiguation module can be placed in front of the constant name. For example:

gin.constant('some.modules.PI', 3.14159)

13. Experiment with multiple Gin files and additional command-line bindings

In many cases, you can define multiple Gin files that contain different parts of the overall configuration of the lab. Additional "tweaks" to the overall configuration can be passed through command-line flags as separate bindings.

A recommended approach is to create a folder containing multiple Gin configurations, and then create an BUILD file containing the following:

filegroup(
    name = "gin_files",
    srcs = glob(["*.gin"]),
    visibility = [":internal"],
)

This filegroup Can be used as a data dependency in a binary file:

data = ["//path/to/configs:gin_files",]

In a binary file, you can define the following flags:

from absl import flags

flags.DEFINE_multi_string(
  'gin_file', None, 'List of paths to the config files.')
flags.DEFINE_multi_string(
  'gin_param', None, 'Newline separated list of Gin parameter bindings.')

FLAGS = flags.FLAGS

Then parse them with Gin:

gin.parse_config_files_and_bindings(FLAGS.gin_file, FLAGS.gin_param)

Finally, the binary file can be run as:

.../run_gin_eval \
  --gin_file=$CONFIGS_PATH/cartpole_balance.gin \
  --gin_file=$CONFIGS_PATH/base_dqn.gin \
  --gin_file=$CONFIGS_PATH/eval.gin \
  --gin_param='evaluate.num_episodes_eval = 10' \
  --gin_param='evaluate.generate_videos = False' \
  --gin_param='evaluate.eval_interval_secs = 60'