U Vc=@spdZddlZddlZddlmZddlmZddlmZddlmZ Gddde Z dd Z d d Z d d ZdS)ztWraps the base_plugin.TBContext to stores additional data shared across API handlers for the HParams plugin backend.N)api_pb2)metadata) json_formatc@seZdZdZdddZddZeddZd d Zdd d Z ddZ ddZ ddZ ddZ ddZddZddZddZd S) ContextaWraps the base_plugin.TBContext to stores additional data shared across API handlers for the HParams plugin backend. Before adding fields to this class, carefully consider whether the field truelly needs to be accessible to all API handlers or if it can be passed separately to the handler constructor. We want to avoid this class becoming a magic container of variables that have no better place. See http://wiki.c2.com/?MagicContainer cCs||_||_dS)aInstantiates a context. Args: tb_context: base_plugin.TBContext. The "base" context we extend. max_domain_discrete_len: int. Only used when computing the experiment from the session runs. The maximum number of disticnt values a string hyperparameter can have for us to populate its 'domain_discrete' field. Typically, only tests should specify a value for this parameter. N) _tb_context_max_domain_discrete_len)self tb_contextZmax_domain_discrete_lenr O/tmp/pip-unpacked-wheel-g8kmtpbc/tensorboard/plugins/hparams/backend_context.py__init__(s zContext.__init__cCs ||}|r|S||||S)aReturns the experiment protobuffer defining the experiment. Accepts a dict containing the plugin contents for all summary tags associated with the hparams plugin, as an optimization for callers who already have this information available, so that this function can minimize its calls to the underlying `DataProvider`. This method first attempts to find a metadata.EXPERIMENT_TAG tag and retrieve the associated protobuffer. If no such tag is found, the method will attempt to build a minimal experiment protobuffer by scanning for all metadata.SESSION_START_INFO_TAG tags (to compute the hparam_infos field of the experiment) and for all scalar tags (to compute the metric_infos field of the experiment). Returns: The experiment protobuffer. If no tags are found from which an experiment protobuffer can be built (possibly, because the event data has not been completely loaded yet), returns an entirely empty experiment. )_find_experiment_tag_compute_experiment_from_runs)r ctx experiment_idhparams_run_to_tag_to_contentZ experimentr r r experiment_from_metadata5s z Context.experiment_from_metadatacCs|jSN)r)r r r r r RszContext.tb_contextcCsdd|DS)NcSs$i|]\}}|dd|DqS)cSsi|]\}}||jqSr )Zplugin_content).0tagZ time_seriesr r r Xsz?Context._convert_plugin_metadata...items)rrunZtag_to_time_seriesr r r rWs z4Context._convert_plugin_metadata..r)r data_provider_outputr r r _convert_plugin_metadataVsz Context._convert_plugin_metadataNcCs||jjj||tj|dS)aReads summary metadata for all hparams time series. Args: experiment_id: String, from `plugin_util.experiment_id`. run_tag_filter: Optional `data.provider.RunTagFilter`, with the semantics as in `list_tensors`. Returns: A dict `d` such that `d[run][tag]` is a `bytes` value with the summary metadata content for the keyed time series. )r plugin_namerun_tag_filter)rr data_providerZ list_tensorsr PLUGIN_NAME)r rrrr r r hparams_metadata_s zContext.hparams_metadatacCs||jjj||tjdS)a&Reads summary metadata for all scalar time series. Args: experiment_id: String, from `plugin_util.experiment_id`. Returns: A dict `d` such that `d[run][tag]` is a `bytes` value with the summary metadata content for the keyed time series. )rr)rrrZ list_scalarsscalar_metadatar )r rrr r r scalars_metadatats zContext.scalars_metadatacCs,|jjj||tj|dd}dd|DS)aReads the most recent values from scalar time series. Args: experiment_id: String. run_tag_filter: Required `data.provider.RunTagFilter`, with the semantics as in `read_scalars`. Returns: A dict `d` such that `d[run][tag]` is a `provider.ScalarDatum` value, with keys only for runs and tags that actually had data, which may be a subset of what was requested. )rrrZ downsamplecSs$i|]\}}|dd|DqS)cSsi|]\}}||dqS)r )rrdatar r r rsz8Context.read_last_scalars...r)rrZ tag_to_datar r r rsz-Context.read_last_scalars..)rrZ read_scalarsr"r r)r rrrrr r r read_last_scalarss  zContext.read_last_scalarscCs4|D]&}|tj}|dk rt|SqdS)zFinds the experiment associcated with the metadata.EXPERIMENT_TAG tag. Returns: The experiment or None if no such experiment is found. N)valuesgetrZEXPERIMENT_TAGZparse_experiment_plugin_data)r rtagsZ maybe_contentr r r rs  zContext._find_experiment_tagcCs0||}|r||||}ng}tj||dS)zComputes a minimal Experiment protocol buffer by scanning the runs. Returns an empty Experiment if there are no hparam infos logged. ) hparam_infos metric_infos)_compute_hparam_infos_compute_metric_infosrZ Experiment)r rrrr+r,r r r rs z%Context._compute_experiment_from_runsc Cstt}|D]B}tj|kr"qt|tj}|jD]\}}|| |qsz;Context._compute_hparam_info_from_values..N) rZ HParamInfoZDATA_TYPE_UNSETsetr:r9DATA_TYPE_STRINGlenrZdomain_discreteextend)r r6r(r8Zdistinct_valuesr<Zv_typer r r r5s,       z(Context._compute_hparam_info_from_valuescCsdd||||DS)Ncss(|] \}}tjtj||ddVqdS))groupr)r6N)rZ MetricInfoZ MetricName)rrrBr r r r=sz0Context._compute_metric_infos..)_compute_metric_names)r rrrr r r r.szContext._compute_metric_infosc stdd|D}t}|||}|D]J\}}t||} | sHq0tj|| dkrbd|fdd|Dq0t|} | | S)aTComputes the list of metric names from all the scalar (run, tag) pairs. The return value is a list of (tag, group) pairs representing the metric names. The list is sorted in Python tuple-order (lexicographical). For example, if the scalar (run, tag) pairs are: ("exp/session1", "loss") ("exp/session2", "loss") ("exp/session2/eval", "loss") ("exp/session2/validation", "accuracy") ("exp/no-session", "loss_2"), and the runs corresponding to sessions are "exp/session1", "exp/session2", this method will return [("loss", ""), ("loss", "/eval"), ("accuracy", "/validation")] More precisely, each scalar (run, tag) pair is converted to a (tag, group) metric name, where group is the suffix of run formed by removing the longest prefix which is a session run. If no session run is a prefix of 'run', the pair is skipped. Returns: A python list containing pairs. Each pair is a (tag, group) pair representing a metric name used in some session. css |]\}}tj|kr|VqdSr)rr2)rrr*r r r r=:s z0Context._compute_metric_names...c3s|]}|fVqdSrr )rrrBr r r=Ls) r>rr#_find_longest_parent_pathospathrelpathupdater1sort) r rrrZ session_runsZmetric_names_setZscalars_run_to_tag_to_contentrr*sessionZmetric_names_listr rFr rCs& zContext._compute_metric_names)r)N)__name__ __module__ __qualname____doc__r rpropertyr rr!r#r'rrr-r5r.rCr r r r rs   !0 rcCs"||kr|sdStj|}q|S)aFinds the longest "parent-path" of 'path' in 'path_set'. This function takes and returns "path-like" strings which are strings made of strings separated by os.sep. No file access is performed here, so these strings need not correspond to actual files in some file-system.. This function returns the longest ancestor path For example, for path_set=["/foo/bar", "/foo", "/bar/foo"] and path="/foo/bar/sub_dir", returns "/foo/bar". Args: path_set: set of path-like strings -- e.g. a list of strings separated by os.sep. No actual disk-access is performed here, so these need not correspond to actual files. path: a path-like string. Returns: The element in path_set which is the longest parent directory of 'path'. N)rHrIdirname)Zpath_setrIr r r rGSs rGcCs4|drtjS|dr tjS|dr0tjSdS)zReturns the type of the google.protobuf.Value message as an api.DataType. Returns None if the type of 'value' is not one of the types supported in api_pb2.DataType. Args: value: google.protobuf.Value message. Z number_value string_valueZ bool_valueN)HasFieldrZDATA_TYPE_FLOAT64r?ZDATA_TYPE_BOOL)r7r r r r:os   r:cCs$t|}|dr |ddS|S)zReturns a string representation of given google.protobuf.Value message. Args: value: google.protobuf.Value message. Assumed to be of type 'number', 'string' or 'bool'. rTr$r%)rZ MessageToJsonrU)r7Z value_in_jsonr r r r;s   r;)rQr/rHZtensorboard.plugins.hparamsrrZgoogle.protobufrZtensorboard.plugins.scalarr"objectrrGr:r;r r r r s    8