mirrors / facebookresearch--audiocraft

项目文件夹

facebookresearch--audiocraft
文件
gh-docs
T
Alexandre Défossez (autodoc) aeffa9f270 api_docs
2025-03-13 16:07:03 +00:00

362 行
21 KiB
HTML
原始文件 永久链接 Blame 文件历史

此文件含有不可见的 Unicode 字符
此文件含有人类无法区分的不可见的 Unicode 字符，但可以由计算机进行不同的处理。 如果您是想特意这样的，可以安全地忽略该警告。 使用 Escape 按钮显示他们。
此文件含有可能会与其他字符混淆的 Unicode 字符。 如果您是想特意这样的，可以安全地忽略该警告。 使用 Escape 按钮显示他们。
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, minimum-scale=1">
<meta name="generator" content="pdoc3 0.11.5">
<title>audiocraft.environment API documentation</title>
<meta name="description" content="Provides cluster and tools configuration across clusters (slurm, dora, utilities).">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/10up-sanitize.css/13.0.0/sanitize.min.css" integrity="sha512-y1dtMcuvtTMJc1yPgEqF0ZjQbhnc/bFhyvIyVNb9Zk5mIGtqVaAB1Ttl28su8AvFMOY0EwRbAe+HCLqj6W7/KA==" crossorigin>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/10up-sanitize.css/13.0.0/typography.min.css" integrity="sha512-Y1DYSb995BAfxobCkKepB1BqJJTPrOp3zPL74AWFugHHmmdcvO+C48WLrUOlhGMc0QG7AE3f7gmvvcrmX2fDoA==" crossorigin>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/default.min.css" crossorigin>
<style>:root{--highlight-color:#fe9}.flex{display:flex !important}body{line-height:1.5em}#content{padding:20px}#sidebar{padding:1.5em;overflow:hidden}#sidebar > *:last-child{margin-bottom:2cm}.http-server-breadcrumbs{font-size:130%;margin:0 0 15px 0}#footer{font-size:.75em;padding:5px 30px;border-top:1px solid #ddd;text-align:right}#footer p{margin:0 0 0 1em;display:inline-block}#footer p:last-child{margin-right:30px}h1,h2,h3,h4,h5{font-weight:300}h1{font-size:2.5em;line-height:1.1em}h2{font-size:1.75em;margin:2em 0 .50em 0}h3{font-size:1.4em;margin:1.6em 0 .7em 0}h4{margin:0;font-size:105%}h1:target,h2:target,h3:target,h4:target,h5:target,h6:target{background:var(--highlight-color);padding:.2em 0}a{color:#058;text-decoration:none;transition:color .2s ease-in-out}a:visited{color:#503}a:hover{color:#b62}.title code{font-weight:bold}h2[id^="header-"]{margin-top:2em}.ident{color:#900;font-weight:bold}pre code{font-size:.8em;line-height:1.4em;padding:1em;display:block}code{background:#f3f3f3;font-family:"DejaVu Sans Mono",monospace;padding:1px 4px;overflow-wrap:break-word}h1 code{background:transparent}pre{border-top:1px solid #ccc;border-bottom:1px solid #ccc;margin:1em 0}#http-server-module-list{display:flex;flex-flow:column}#http-server-module-list div{display:flex}#http-server-module-list dt{min-width:10%}#http-server-module-list p{margin-top:0}.toc ul,#index{list-style-type:none;margin:0;padding:0}#index code{background:transparent}#index h3{border-bottom:1px solid #ddd}#index ul{padding:0}#index h4{margin-top:.6em;font-weight:bold}@media (min-width:200ex){#index .two-column{column-count:2}}@media (min-width:300ex){#index .two-column{column-count:3}}dl{margin-bottom:2em}dl dl:last-child{margin-bottom:4em}dd{margin:0 0 1em 3em}#header-classes + dl > dd{margin-bottom:3em}dd dd{margin-left:2em}dd p{margin:10px 0}.name{background:#eee;font-size:.85em;padding:5px 10px;display:inline-block;min-width:40%}.name:hover{background:#e0e0e0}dt:target .name{background:var(--highlight-color)}.name > span:first-child{white-space:nowrap}.name.class > span:nth-child(2){margin-left:.4em}.inherited{color:#999;border-left:5px solid #eee;padding-left:1em}.inheritance em{font-style:normal;font-weight:bold}.desc h2{font-weight:400;font-size:1.25em}.desc h3{font-size:1em}.desc dt code{background:inherit}.source > summary,.git-link-div{color:#666;text-align:right;font-weight:400;font-size:.8em;text-transform:uppercase}.source summary > *{white-space:nowrap;cursor:pointer}.git-link{color:inherit;margin-left:1em}.source pre{max-height:500px;overflow:auto;margin:0}.source pre code{font-size:12px;overflow:visible;min-width:max-content}.hlist{list-style:none}.hlist li{display:inline}.hlist li:after{content:',\2002'}.hlist li:last-child:after{content:none}.hlist .hlist{display:inline;padding-left:1em}img{max-width:100%}td{padding:0 .5em}.admonition{padding:.1em 1em;margin:1em 0}.admonition-title{font-weight:bold}.admonition.note,.admonition.info,.admonition.important{background:#aef}.admonition.todo,.admonition.versionadded,.admonition.tip,.admonition.hint{background:#dfd}.admonition.warning,.admonition.versionchanged,.admonition.deprecated{background:#fd4}.admonition.error,.admonition.danger,.admonition.caution{background:lightpink}</style>
<style media="screen and (min-width: 700px)">@media screen and (min-width:700px){#sidebar{width:30%;height:100vh;overflow:auto;position:sticky;top:0}#content{width:70%;max-width:100ch;padding:3em 4em;border-left:1px solid #ddd}pre code{font-size:1em}.name{font-size:1em}main{display:flex;flex-direction:row-reverse;justify-content:flex-end}.toc ul ul,#index ul ul{padding-left:1em}.toc > ul > li{margin-top:.5em}}</style>
<style media="print">@media print{#sidebar h1{page-break-before:always}.source{display:none}}@media print{*{background:transparent !important;color:#000 !important;box-shadow:none !important;text-shadow:none !important}a[href]:after{content:" (" attr(href) ")";font-size:90%}a[href][title]:after{content:none}abbr[title]:after{content:" (" attr(title) ")"}.ir a:after,a[href^="javascript:"]:after,a[href^="#"]:after{content:""}pre,blockquote{border:1px solid #999;page-break-inside:avoid}thead{display:table-header-group}tr,img{page-break-inside:avoid}img{max-width:100% !important}@page{margin:0.5cm}p,h2,h3{orphans:3;widows:3}h1,h2,h3,h4,h5,h6{page-break-after:avoid}}</style>
<script defer src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js" integrity="sha512-D9gUyxqja7hBtkWpPWGt9wfbfaMGVt9gnyCvYa+jojwwPHLCzUm5i8rpk7vD7wNee9bA35eYIjobYPaQuKS1MQ==" crossorigin></script>
<script>window.addEventListener('DOMContentLoaded', () => {
hljs.configure({languages: ['bash', 'css', 'diff', 'graphql', 'ini', 'javascript', 'json', 'plaintext', 'python', 'python-repl', 'rust', 'shell', 'sql', 'typescript', 'xml', 'yaml']});
hljs.highlightAll();
/* Collapse source docstrings */
setTimeout(() => {
[...document.querySelectorAll('.hljs.language-python > .hljs-string')]
.filter(el => el.innerHTML.length > 200 && ['"""', "'''"].includes(el.innerHTML.substring(0, 3)))
.forEach(el => {
let d = document.createElement('details');
d.classList.add('hljs-string');
d.innerHTML = '<summary>"""</summary>' + el.innerHTML.substring(3);
el.replaceWith(d);
});
}, 100);
})</script>
</head>
<body>
<main>
<article id="content">
<header>
<h1 class="title">Module <code>audiocraft.environment</code></h1>
</header>
<section id="section-intro">
<p>Provides cluster and tools configuration across clusters (slurm, dora, utilities).</p>
</section>
<section>
</section>
<section>
</section>
<section>
</section>
<section>
<h2 class="section-title" id="header-classes">Classes</h2>
<dl>
<dt id="audiocraft.environment.AudioCraftEnvironment"><code class="flex name class">
<span>class <span class="ident">AudioCraftEnvironment</span></span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">class AudioCraftEnvironment:
&#34;&#34;&#34;Environment configuration for teams and clusters.
AudioCraftEnvironment picks compute cluster settings (slurm, dora) from the current running environment
or declared variable and the loaded team configuration. Additionally, the AudioCraftEnvironment
provides pointers to a reference folder resolved automatically across clusters that is shared across team members,
allowing to share sigs or other files to run jobs. Finally, it provides dataset mappers to automatically
map dataset file paths to new locations across clusters, allowing to use the same manifest of files across cluters.
The cluster type is identified automatically and base configuration file is read from config/teams.yaml.
Use the following environment variables to specify the cluster, team or configuration:
AUDIOCRAFT_CLUSTER (optional): Cluster type to enforce. Useful if the cluster type
cannot be inferred automatically.
AUDIOCRAFT_CONFIG (optional): Path to yaml config holding the teams configuration.
If not set, configuration is read from config/teams.yaml.
AUDIOCRAFT_TEAM (optional): Name of the team. Recommended to set to your own team.
Cluster configuration are shared across teams to match compute allocation,
specify your cluster configuration in the configuration file under a key mapping
your team name.
&#34;&#34;&#34;
_instance = None
DEFAULT_TEAM = &#34;default&#34;
def __init__(self) -&gt; None:
&#34;&#34;&#34;Loads configuration.&#34;&#34;&#34;
self.team: str = os.getenv(&#34;AUDIOCRAFT_TEAM&#34;, self.DEFAULT_TEAM)
cluster_type = _guess_cluster_type()
cluster = os.getenv(
&#34;AUDIOCRAFT_CLUSTER&#34;, cluster_type.value
)
logger.info(&#34;Detecting cluster type %s&#34;, cluster_type)
self.cluster: str = cluster
config_path = os.getenv(
&#34;AUDIOCRAFT_CONFIG&#34;,
Path(__file__)
.parent.parent.joinpath(&#34;config/teams&#34;, self.team)
.with_suffix(&#34;.yaml&#34;),
)
self.config = omegaconf.OmegaConf.load(config_path)
self._dataset_mappers = []
cluster_config = self._get_cluster_config()
if &#34;dataset_mappers&#34; in cluster_config:
for pattern, repl in cluster_config[&#34;dataset_mappers&#34;].items():
regex = re.compile(pattern)
self._dataset_mappers.append((regex, repl))
def _get_cluster_config(self) -&gt; omegaconf.DictConfig:
assert isinstance(self.config, omegaconf.DictConfig)
return self.config[self.cluster]
@classmethod
def instance(cls):
if cls._instance is None:
cls._instance = cls()
return cls._instance
@classmethod
def reset(cls):
&#34;&#34;&#34;Clears the environment and forces a reload on next invocation.&#34;&#34;&#34;
cls._instance = None
@classmethod
def get_team(cls) -&gt; str:
&#34;&#34;&#34;Gets the selected team as dictated by the AUDIOCRAFT_TEAM env var.
If not defined, defaults to &#34;labs&#34;.
&#34;&#34;&#34;
return cls.instance().team
@classmethod
def get_cluster(cls) -&gt; str:
&#34;&#34;&#34;Gets the detected cluster.
This value can be overridden by the AUDIOCRAFT_CLUSTER env var.
&#34;&#34;&#34;
return cls.instance().cluster
@classmethod
def get_dora_dir(cls) -&gt; Path:
&#34;&#34;&#34;Gets the path to the dora directory for the current team and cluster.
Value is overridden by the AUDIOCRAFT_DORA_DIR env var.
&#34;&#34;&#34;
cluster_config = cls.instance()._get_cluster_config()
dora_dir = os.getenv(&#34;AUDIOCRAFT_DORA_DIR&#34;, cluster_config[&#34;dora_dir&#34;])
logger.warning(f&#34;Dora directory: {dora_dir}&#34;)
return Path(dora_dir)
@classmethod
def get_reference_dir(cls) -&gt; Path:
&#34;&#34;&#34;Gets the path to the reference directory for the current team and cluster.
Value is overridden by the AUDIOCRAFT_REFERENCE_DIR env var.
&#34;&#34;&#34;
cluster_config = cls.instance()._get_cluster_config()
return Path(os.getenv(&#34;AUDIOCRAFT_REFERENCE_DIR&#34;, cluster_config[&#34;reference_dir&#34;]))
@classmethod
def get_slurm_exclude(cls) -&gt; tp.Optional[str]:
&#34;&#34;&#34;Get the list of nodes to exclude for that cluster.&#34;&#34;&#34;
cluster_config = cls.instance()._get_cluster_config()
return cluster_config.get(&#34;slurm_exclude&#34;)
@classmethod
def get_slurm_partitions(cls, partition_types: tp.Optional[tp.List[str]] = None) -&gt; str:
&#34;&#34;&#34;Gets the requested partitions for the current team and cluster as a comma-separated string.
Args:
partition_types (list[str], optional): partition types to retrieve. Values must be
from [&#39;global&#39;, &#39;team&#39;]. If not provided, the global partition is returned.
&#34;&#34;&#34;
if not partition_types:
partition_types = [&#34;global&#34;]
cluster_config = cls.instance()._get_cluster_config()
partitions = [
cluster_config[&#34;partitions&#34;][partition_type]
for partition_type in partition_types
]
return &#34;,&#34;.join(partitions)
@classmethod
def resolve_reference_path(cls, path: tp.Union[str, Path]) -&gt; Path:
&#34;&#34;&#34;Converts reference placeholder in path with configured reference dir to resolve paths.
Args:
path (str or Path): Path to resolve.
Returns:
Path: Resolved path.
&#34;&#34;&#34;
path = str(path)
if path.startswith(&#34;//reference&#34;):
reference_dir = cls.get_reference_dir()
logger.warn(f&#34;Reference directory: {reference_dir}&#34;)
assert (
reference_dir.exists() and reference_dir.is_dir()
), f&#34;Reference directory does not exist: {reference_dir}.&#34;
path = re.sub(&#34;^//reference&#34;, str(reference_dir), path)
return Path(path)
@classmethod
def apply_dataset_mappers(cls, path: str) -&gt; str:
&#34;&#34;&#34;Applies dataset mapping regex rules as defined in the configuration.
If no rules are defined, the path is returned as-is.
&#34;&#34;&#34;
instance = cls.instance()
for pattern, repl in instance._dataset_mappers:
path = pattern.sub(repl, path)
return path</code></pre>
</details>
<div class="desc"><p>Environment configuration for teams and clusters.</p>
<p>AudioCraftEnvironment picks compute cluster settings (slurm, dora) from the current running environment
or declared variable and the loaded team configuration. Additionally, the AudioCraftEnvironment
provides pointers to a reference folder resolved automatically across clusters that is shared across team members,
allowing to share sigs or other files to run jobs. Finally, it provides dataset mappers to automatically
map dataset file paths to new locations across clusters, allowing to use the same manifest of files across cluters.</p>
<p>The cluster type is identified automatically and base configuration file is read from config/teams.yaml.
Use the following environment variables to specify the cluster, team or configuration:</p>
<pre><code>AUDIOCRAFT_CLUSTER (optional): Cluster type to enforce. Useful if the cluster type
cannot be inferred automatically.
AUDIOCRAFT_CONFIG (optional): Path to yaml config holding the teams configuration.
If not set, configuration is read from config/teams.yaml.
AUDIOCRAFT_TEAM (optional): Name of the team. Recommended to set to your own team.
Cluster configuration are shared across teams to match compute allocation,
specify your cluster configuration in the configuration file under a key mapping
your team name.
</code></pre>
<p>Loads configuration.</p></div>
<h3>Class variables</h3>
<dl>
<dt id="audiocraft.environment.AudioCraftEnvironment.DEFAULT_TEAM"><code class="name">var <span class="ident">DEFAULT_TEAM</span></code></dt>
<dd>
<div class="desc"></div>
</dd>
</dl>
<h3>Static methods</h3>
<dl>
<dt id="audiocraft.environment.AudioCraftEnvironment.apply_dataset_mappers"><code class="name flex">
<span>def <span class="ident">apply_dataset_mappers</span></span>(<span>path: str) ‑> str</span>
</code></dt>
<dd>
<div class="desc"><p>Applies dataset mapping regex rules as defined in the configuration.
If no rules are defined, the path is returned as-is.</p></div>
</dd>
<dt id="audiocraft.environment.AudioCraftEnvironment.get_cluster"><code class="name flex">
<span>def <span class="ident">get_cluster</span></span>(<span>) ‑> str</span>
</code></dt>
<dd>
<div class="desc"><p>Gets the detected cluster.
This value can be overridden by the AUDIOCRAFT_CLUSTER env var.</p></div>
</dd>
<dt id="audiocraft.environment.AudioCraftEnvironment.get_dora_dir"><code class="name flex">
<span>def <span class="ident">get_dora_dir</span></span>(<span>) ‑> pathlib.Path</span>
</code></dt>
<dd>
<div class="desc"><p>Gets the path to the dora directory for the current team and cluster.
Value is overridden by the AUDIOCRAFT_DORA_DIR env var.</p></div>
</dd>
<dt id="audiocraft.environment.AudioCraftEnvironment.get_reference_dir"><code class="name flex">
<span>def <span class="ident">get_reference_dir</span></span>(<span>) ‑> pathlib.Path</span>
</code></dt>
<dd>
<div class="desc"><p>Gets the path to the reference directory for the current team and cluster.
Value is overridden by the AUDIOCRAFT_REFERENCE_DIR env var.</p></div>
</dd>
<dt id="audiocraft.environment.AudioCraftEnvironment.get_slurm_exclude"><code class="name flex">
<span>def <span class="ident">get_slurm_exclude</span></span>(<span>) ‑> str | None</span>
</code></dt>
<dd>
<div class="desc"><p>Get the list of nodes to exclude for that cluster.</p></div>
</dd>
<dt id="audiocraft.environment.AudioCraftEnvironment.get_slurm_partitions"><code class="name flex">
<span>def <span class="ident">get_slurm_partitions</span></span>(<span>partition_types: List[str] | None = None) ‑> str</span>
</code></dt>
<dd>
<div class="desc"><p>Gets the requested partitions for the current team and cluster as a comma-separated string.</p>
<h2 id="args">Args</h2>
<dl>
<dt><strong><code>partition_types</code></strong> :&ensp;<code>list[str]</code>, optional</dt>
<dd>partition types to retrieve. Values must be
from ['global', 'team']. If not provided, the global partition is returned.</dd>
</dl></div>
</dd>
<dt id="audiocraft.environment.AudioCraftEnvironment.get_team"><code class="name flex">
<span>def <span class="ident">get_team</span></span>(<span>) ‑> str</span>
</code></dt>
<dd>
<div class="desc"><p>Gets the selected team as dictated by the AUDIOCRAFT_TEAM env var.
If not defined, defaults to "labs".</p></div>
</dd>
<dt id="audiocraft.environment.AudioCraftEnvironment.instance"><code class="name flex">
<span>def <span class="ident">instance</span></span>(<span>)</span>
</code></dt>
<dd>
<div class="desc"></div>
</dd>
<dt id="audiocraft.environment.AudioCraftEnvironment.reset"><code class="name flex">
<span>def <span class="ident">reset</span></span>(<span>)</span>
</code></dt>
<dd>
<div class="desc"><p>Clears the environment and forces a reload on next invocation.</p></div>
</dd>
<dt id="audiocraft.environment.AudioCraftEnvironment.resolve_reference_path"><code class="name flex">
<span>def <span class="ident">resolve_reference_path</span></span>(<span>path: str | pathlib.Path) ‑> pathlib.Path</span>
</code></dt>
<dd>
<div class="desc"><p>Converts reference placeholder in path with configured reference dir to resolve paths.</p>
<h2 id="args">Args</h2>
<dl>
<dt><strong><code>path</code></strong> :&ensp;<code>str</code> or <code>Path</code></dt>
<dd>Path to resolve.</dd>
</dl>
<h2 id="returns">Returns</h2>
<dl>
<dt><code>Path</code></dt>
<dd>Resolved path.</dd>
</dl></div>
</dd>
</dl>
</dd>
</dl>
</section>
</article>
<nav id="sidebar">
<div class="toc">
<ul></ul>
</div>
<ul id="index">
<li><h3>Super-module</h3>
<ul>
<li><code><a title="audiocraft" href="index.html">audiocraft</a></code></li>
</ul>
</li>
<li><h3><a href="#header-classes">Classes</a></h3>
<ul>
<li>
<h4><code><a title="audiocraft.environment.AudioCraftEnvironment" href="#audiocraft.environment.AudioCraftEnvironment">AudioCraftEnvironment</a></code></h4>
<ul class="">
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.DEFAULT_TEAM" href="#audiocraft.environment.AudioCraftEnvironment.DEFAULT_TEAM">DEFAULT_TEAM</a></code></li>
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.apply_dataset_mappers" href="#audiocraft.environment.AudioCraftEnvironment.apply_dataset_mappers">apply_dataset_mappers</a></code></li>
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.get_cluster" href="#audiocraft.environment.AudioCraftEnvironment.get_cluster">get_cluster</a></code></li>
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.get_dora_dir" href="#audiocraft.environment.AudioCraftEnvironment.get_dora_dir">get_dora_dir</a></code></li>
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.get_reference_dir" href="#audiocraft.environment.AudioCraftEnvironment.get_reference_dir">get_reference_dir</a></code></li>
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.get_slurm_exclude" href="#audiocraft.environment.AudioCraftEnvironment.get_slurm_exclude">get_slurm_exclude</a></code></li>
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.get_slurm_partitions" href="#audiocraft.environment.AudioCraftEnvironment.get_slurm_partitions">get_slurm_partitions</a></code></li>
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.get_team" href="#audiocraft.environment.AudioCraftEnvironment.get_team">get_team</a></code></li>
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.instance" href="#audiocraft.environment.AudioCraftEnvironment.instance">instance</a></code></li>
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.reset" href="#audiocraft.environment.AudioCraftEnvironment.reset">reset</a></code></li>
<li><code><a title="audiocraft.environment.AudioCraftEnvironment.resolve_reference_path" href="#audiocraft.environment.AudioCraftEnvironment.resolve_reference_path">resolve_reference_path</a></code></li>
</ul>
</li>
</ul>
</li>
</ul>
</nav>
</main>
<footer id="footer">
<p>Generated by <a href="https://pdoc3.github.io/pdoc" title="pdoc: Python API documentation generator"><cite>pdoc</cite> 0.11.5</a>.</p>
</footer>
</body>
</html>
在新工单中引用 查看 Git Blame 复制永久链接