U
Vcl @ s d Z ddlZddlZddlZddlZddlZddlZddlZddlm Z ddlm
Z
ddl m Z ddl
mZ ddlmZ ddlmZ dd lmZ dd
l mZ e Zd Zd d
gZdZG dd dejZG dd dej Z dd Z dd Z dd Z dS )z TensorBoard core plugin package. N)utils)wrappers) plugin_util) http_util) base_plugin) grpc_util)
tb_logging)versioniv ztext/javascriptzapplication/javascriptQ c @ s e Zd ZdZdZd&ddZdd Zdd Zd
d Ze j
j d d
Z e j
j dd Z
e j
j dd Ze j
j dd Ze j
j dd Zdd Ze j
j dd Ze j
j dd Ze j
j d d Ze j
j d d Zd d! Ze j
j d"d# Ze j
j d$d% ZdS )'
CorePluginzCore plugin for TensorBoard.
This plugin serves runs, configuration data, and static assets. This
plugin should always be present in a TensorBoard WSGI application.
coreNc C s` |j | _|j r|j jnd}|jp"|| _|j| _|j r<|j jnd| _|j | _
|j | _ t
|| _dS )aj Instantiates CorePlugin.
Args:
context: A base_plugin.TBContext instance.
include_debug_info: If true, `/data/environment` will include some
basic information like the TensorBoard server version. Disabled by
default to prevent surprising information leaks in custom builds of
TensorBoard.
N)flags_flags logdir_speclogdir_logdir window_title
_window_title path_prefix _path_prefixZassets_zip_provider_assets_zip_provider
data_provider_data_providerbool_include_debug_info)selfcontextinclude_debug_infor r H/tmp/pip-unpacked-wheel-g8kmtpbc/tensorboard/plugins/core/core_plugin.py__init__; s
zCorePlugin.__init__c C s dS )NTr r r r r is_activeN s zCorePlugin.is_activec C sP | j | j| j| j| j| j| j| j| j| j| j | j| j| jd}| |
|S )N)z/___rPc_sWiTcH___z/audioz/data/environmentz /data/logdirz
/data/runsz/data/experimentsz/data/experiment_runsz/data/notificationsz/data/window_propertiesz/eventsz /favicon.icoz/graphsz /histogramsz/images) _send_404_without_logging_redirect_to_index_serve_environment
_serve_logdir _serve_runs_serve_experiments_serve_experiment_runs_serve_notifications_serve_window_propertiesupdateget_resource_apps)r appsr r r get_plugin_appsQ s" zCorePlugin.get_plugin_appsc
C s i }| j s|S | x}t|b}| D ]R}||}|dkrZt| j||d| < q,t|}t| j ||}||d| < q,W 5 Q R X W 5 Q R X |d |d< |S )Nz
index.html/z /index.html)
r zipfileZipFilenamelistread functoolspartial _serve_index_gzip _serve_asset)r r/ fpZzip_pathcontentgzipped_asset_bytesZwsgi_appr r r r. e s,
" z CorePlugin.get_resource_appsc C s t j|ddddS )Nz Not foundz
text/plaini )coder Respondr requestr r r r$ } s z$CorePlugin._send_404_without_loggingc C s
t dS )Nr1 )r redirect)r Zunused_requestr r r r% s z CorePlugin._redirect_to_indexc C sB t |d pd}|jdr*|tkr*tnd}tj|||d|dS )z4Serves a pre-gzipped static asset from the zip file.r zapplication/octet-streamZ
_file_hashgzip)content_encodingexpires) mimetypes
guess_typeargsget JS_MIMETYPESJS_CACHE_EXPIRATION_IN_SECSr rA )r r< r> rC mimetyperG r r r r: s
zCorePlugin._serve_assetc C sB | j rt| j |jnd}d| }|d| }tj||dddS )a Serves index.html content.
Note that we opt out of gzipping index.html to write preamble before the
resource content. This inflates the resource size from 2x kiB to 1xx
kiB, but we require an ability to flush preamble with the HTML content.
.z;zutf-8z text/htmlidentity)rF )r posixpathrelpathZ script_rootencoder rA )r Zindex_asset_bytesrC rR Z meta_headerr= r r r r8 s
zCorePlugin._serve_indexc C sr t |j}t |j}| jj||d}tj|j| j |j
|j |j d}| j
rdt| j| d|d< t||dS )z:Serve a JSON object describing the TensorBoard parameters.
experiment_id)r
data_locationr experiment_nameexperiment_description
creation_time)r r debugapplication/json)r r environrU r Zexperiment_metadatar VERSIONrV r rW rX rY r str
_render_flagsr rA )r rC ctx
experimentZmd environmentr r r r& s,
z CorePlugin._serve_environmentc s( | j dkrdS fdd t| j S )zReturn a JSON-and-human-friendly version of `self._flags`.
Like `json.loads(json.dumps(self._flags, default=str))` but
without the wasteful serialization overhead.
Nc sb t | td tttfr| S t | ttfr: fdd| D S t | trZ fdd| D S t| S )Nc s g | ] } |qS r r ).0vgor r
s z8CorePlugin._render_flags..go..c s i | ]\}}t | |qS r )r^ )rc krd re r r
s z8CorePlugin._render_flags..go..)
isinstancetyper^ intfloatlisttupledictitems)xre r r rf s
z$CorePlugin._render_flags..go)r varsr" r re r r_ s
zCorePlugin._render_flagsc C s t |d| jidS )z@Respond with a JSON object containing this TensorBoard's logdir.r r[ )r rA r rB r r r r' s
zCorePlugin._serve_logdirc C s t |d| jidS )zLServe a JSON object containing this TensorBoard's window
properties.r r[ )r rA r rB r r r r, s
z#CorePlugin._serve_window_propertiesc C sP t |j}t |j}t| jj||ddd d}dd |D }t||dS )zServe a JSON array of run names, ordered by run started time.
Sort order is by started time (aka first event time) with empty
times sorted last, and then ties are broken by sorting on the
run name.
rT c S s | j d k r| j ntd| jfS )Ninf)
start_timerm run_name)runr r r s z(CorePlugin._serve_runs..)keyc S s g | ]
}|j qS r )rv )rc rw r r r rg s z*CorePlugin._serve_runs..r[ ) r r r\ rU sortedr Z list_runsr rA )r rC r` ra runsZ run_namesr r r r( s zCorePlugin._serve_runsc C s | }t||dS )zServe a JSON array of experiments.
Experiments are ordered by experiment started time (aka first
event time) with empty times sorted last, and then ties are
broken by sorting on the experiment name.
r[ )list_experiments_implr rA r rC resultsr r r r)
s z CorePlugin._serve_experimentsc C s g S Nr r" r r r r| s z CorePlugin.list_experiments_implc C s g }t ||dS )a Serve a JSON runs of an experiment, specified with query param
`experiment`, with their nested data, tag, populated.
Runs returned are ordered by started time (aka first event time)
with empty times sorted last, and then ties are broken by
sorting on the run name. Tags are sorted by its name,
displayName, and lastly, inserted time.
r[ r@ r} r r r r* s
z!CorePlugin._serve_experiment_runsc C s t d}d|_|S )z6Serve JSON payload of notifications to show in the UI.z../notifications_note.jsonF)r rD Zautocorrect_location_header)r rC responser r r r+ % s
z CorePlugin._serve_notifications)N)__name__
__module__ __qualname____doc__Z plugin_namer! r# r0 r. r RequestZ applicationr$ r% r: r8 r& r_ r' r, r( r) r| r* r+ r r r r r 2 s<
r c @ s2 e Zd ZdZd ddZdd Zdd Zd d
ZdS ) CorePluginLoaderzCorePlugin factory.Nc C s
|| _ d S r )r )r r r r r r! 4 s zCorePluginLoader.__init__c C s` |j ddtddd |j ddtddd |j d d
td d d |j d
ddd |j dddd ddt d |j dddd ddgdd d |j d td d d d!gd"d# |j d$tdd%d& |j d'tjtjjtj d(d# |j d)dtdd*d |j d+dd,d ddgdd-d |j d.d/tdd0d |j d1dd2d |j d3dd4d |j d5dd6d |j d7d8tdd9d |j d:dtdd;d |j dd?tdd@d |j dAdBtdCdDd |j dEdFtdGdHd |j dIdJtd d dKdLdMgdNdO |j dPddQd ddgd dRd |j dSdFtdTdUd |j dVdJtd d d d!gdWdO |j dXt ddYd& |j dZdd[d ddgd d\d d S )]z.Adds standard TensorBoard CLI flags to parser.z--logdirPATHr
a Directory where TensorBoard will look to find TensorFlow event files
that it can display. TensorBoard will recursively walk the directory
structure rooted at logdir, looking for .*tfevents.* files.
A leading tilde will be expanded with the semantics of Python's
os.expanduser function.
)metavarrk defaulthelpz
--logdir_specZ PATH_SPECa Like `--logdir`, but with special interpretation for commas and colons:
commas separate multiple runs, where a colon specifies a new name for a
run. For example:
`tensorboard --logdir_spec=name1:/path/to/logs/1,name2:/path/to/logs/2`.
This flag is discouraged and can usually be avoided. TensorBoard walks
log directories recursively; for finer-grained control, prefer using a
symlink tree. Some features may not work when using `--logdir_spec`
instead of `--logdir`.
z--hostZADDRNzWhat host to listen to (default: localhost). To serve to the entire local
network on both IPv4 and IPv6, see `--bind_all`, with which this option is
mutually exclusive.
z
--bind_all
store_truezServe on all public interfaces. This will expose your TensorBoard instance to
the network on both IPv4 and IPv6 (where available). Mutually exclusive with
`--host`.
)actionr z--portZPORTc S s | dkr d S t | S )Nr )rl )sr r r rx s z/CorePluginLoader.define_flags..r zPort to serve TensorBoard on. Pass 0 to request an unused port selected
by the operating system, or pass "default" to try to bind to the default
port (%s) but search for a nearby free port if the default port is
unavailable. (default: "default").z --reuse_portZBOOLc S s ddd | | S NTF)truefalserK lowerrd r r r rx r TFa Enables the SO_REUSEPORT option on the socket opened by TensorBoard's HTTP
server, for platforms that support it. This is useful in cases when a parent
process has obtained the port already and wants to delegate access to the
port to TensorBoard as a subprocess.(default: %(default)s).)r rk choicesr r z --load_fastautor r a Use alternate mechanism to load data. Typically 100x faster or more, but only
available on some platforms and invocations. Defaults to "auto" to use this new
mode only if available, otherwise falling back to the legacy loading path. Set
to "true" to suppress the advisory note and hard-fail if the fast codepath is
not available. Set to "false" to always fall back. Feedback/issues:
https://github.com/tensorflow/tensorboard/issues/4784
(default: %(default)s)
)rk r r r z--extra_data_server_flagszExperimental. With `--load_fast`, pass these additional command-line flags to
the data server. Subject to POSIX word splitting per `shlex.split`. Meant for
debugging; not officially supported.
)rk r r z--grpc_creds_typezcExperimental. The type of credentials to use to connect to the data server.
(default: %(default)s)
z--grpc_data_providerzxExperimental. Address of a gRPC server exposing a data provider. Set to empty
string to disable. (default: %(default)s)
z--purge_orphaned_datac S s ddd | | S r r r r r r rx r zWhether to purge data that may have been orphaned due to TensorBoard
restarts. Setting --purge_orphaned_data=False can be used to debug data
disappearance. (default: %(default)s)z--dbURIzw[experimental] sets SQL database URI and enables DB backend mode, which is
read-only unless --db_import is also passed.z --db_importz[experimental] enables DB read-and-import mode, which in combination with
--logdir imports event files into a DB backend on the fly. The backing DB is
temporary unless --db is also passed to specify a DB path to use.z --inspectaO Prints digests of event files to command line.
This is useful when no data is shown on TensorBoard, or the data shown
looks weird.
Must specify one of `logdir` or `event_file` flag.
Example usage:
`tensorboard --inspect --logdir mylogdir --tag loss`
See tensorboard/backend/event_processing/event_file_inspector.py for more info.z --version_tbz!Prints the version of Tensorboardz--tagZTAGz%tag to query for; used with --inspectz --event_filezhThe particular event file to query for. Only used if --inspect is
present and --logdir is not specified.z
--path_prefixa An optional, relative prefix to the path, e.g. "/path/to/tensorboard".
resulting in the new base url being located at
localhost:6006/path/to/tensorboard under default settings. A leading
slash is required when specifying the path_prefix. A trailing slash is
optional and has no effect. The path_prefix can be leveraged for path
based routing of an ELB when the website base_url is not available e.g.
"example.site.com/path/to/tensorboard/".z--window_titleZTEXTz changes title of browser windowz--max_reload_threadsZCOUNT zThe max number of threads that TensorBoard can use to reload runs. Not
relevant for db read-only mode. Each thread reloads one run at a time.
(default: %(default)s)z--reload_intervalZSECONDSg @zHow often the backend should load more data, in seconds. Set to 0 to
load just once at startup. Must be non-negative. (default: %(default)s)z
--reload_taskZTYPEthreadprocessblockingas [experimental] The mechanism to use for the background data reload task.
The default "auto" option will conditionally use threads for legacy reloading
and a child process for DB import reloading. The "process" option is only
useful with DB import mode. The "blocking" option will block startup until
reload finishes, and requires --load_interval=0. (default: %(default)s))r rk r r r z--reload_multifilec S s ddd | | S r r r r r r rx Q r a [experimental] If true, this enables experimental support for continuously
polling multiple event files in each run directory for newly appended data
(rather than only polling the last event file). Event files will only be
polled as long as their most recently read data is newer than the threshold
defined by --reload_multifile_inactive_secs, to limit resource usage. Beware
of running out of memory if the logdir contains many active event files.
(default: false)z --reload_multifile_inactive_secsr
a8 [experimental] Configures the age threshold in seconds at which an event file
that has no event wall time more recent than that will be considered an
inactive file and no longer polled (to limit resource usage). If set to -1,
no maximum age will be enforced, but beware of running out of memory and
heavier filesystem read traffic. If set to 0, this reverts to the older
last-file-only polling strategy (akin to --reload_multifile=false).
(default: %(default)s - intended to ensure an event file remains active if
it receives new data at least once per 24 hour period)z--generic_dataa] [experimental] Hints whether plugins should read from generic data
provider infrastructure. For plugins that support only the legacy
multiplexer APIs or only the generic data APIs, this option has no
effect. The "auto" option enables this only for plugins that are
considered to have stable support for generic data providers. (default:
%(default)s)z--samples_per_plugina> An optional comma separated list of plugin_name=num_samples pairs to
explicitly specify how many samples to keep per tag for that plugin. For
unspecified plugins, TensorBoard randomly downsamples logged summaries
to reasonable values to prevent out-of-memory errors for long running
jobs. This flag allows fine control over that downsampling. Note that if a
plugin is not specified in this list, a plugin-specific default number of
samples will be enforced. (for example, 10 for images, 500 for histograms,
and 1000 for scalars). Most users should not need to set this flag.z--detect_file_replacementc S s ddd | | S r r r r r r rx r aQ [experimental] If true, this enables experimental support for detecting when
event files are replaced with new versions that contain additional data. This is
not needed in the normal case where new data is either appended to an existing
file or written to a brand new file, but it arises, for example, when using
rsync without the --inplace option, in which new versions of the original file
are first written to a temporary file, then swapped into the final location.
This option is currently incompatible with --load_fast=true, and if passed will
disable fast-loading mode. (default: false))
add_argumentr^ DEFAULT_PORTr ZChannelCredsTypeZLOCALr rl _nonnegative_float_parse_samples_per_plugin)r parserr r r define_flags7 sv
z CorePluginLoader.define_flagsc C s t j}|jrn|jrL|jr"|d|jr6|jr6|d|js|js|dnn|jrb|jrb|dnX|js|js|js|js|dn6|j dk r|j
r|dn |j dkr|j d kr|d
|j
d |_
|j
r|j
d s|d |j
dS )
z/Fixes standard TensorBoard CLI flags to parser.z.--logdir_spec is not supported with --inspect.z;Must specify either --logdir or --event_file, but not both.z-Must specify either --logdir or --event_file.z/May not specify both --logdir and --logdir_speczA logdir or db must be specified. For example `tensorboard --logdir mylogdir` or `tensorboard --db sqlite:~/.tensorboard.db`. Run `tensorboard --helpfull` for details and examples.Nz,Must not specify both --host and --bind_all.r TzHMust not specify both --load_fast=true and--detect_file_replacement=truer1 z/Path prefix must start with slash, but got: %r.)r
FlagsErrorZ
version_tbinspectr r Z
event_filedbZgrpc_data_providerhostZbind_allZ load_fastZdetect_file_replacementr rstrip
startswith)r r r r r r fix_flags sZ
zCorePluginLoader.fix_flagsc C s t || jdS )z Creates CorePlugin instance.)r )r r )r r r r r load s zCorePluginLoader.load)N)r r r r r! r r r r r r r r 1 s
o1r c C s8 t }tj|dddd}|| W 5 Q R X | S )Nwb r )fileobjmode
compresslevelmtime)ioBytesIOrE GzipFilewritegetvalue)Z
bytestringoutfr r r r9 s r9 c C s: i }| dD ]&}|r| d\}}t|||< q|S )zCParses `value` as a string-to-int dict in the form `foo=12,bar=34`.,=)splitstriprl )valueresulttokenrh rd r r r r s r c C sJ z t | } W n" tk
r. td| Y nX | dksFtd| | S )Nzinvalid float: %rr zmust be non-negative: %r)rm
ValueErrorargparseArgumentTypeErrorr r r r r s r )!r r r6 rE r rH rQ r2 Zwerkzeugr r Z tensorboardr Ztensorboard.backendr Ztensorboard.pluginsr Ztensorboard.utilr r r Z
get_loggerloggerr rL rM ZTBPluginr ZTBLoaderr r9 r r r r r r s: ,