{ "cells": [ { "cell_type": "markdown", "id": "ce0fa061-3f49-46c3-ba5c-8dcca7d283d3", "metadata": {}, "source": [ "# 3.4 功能预测任务" ] }, { "cell_type": "markdown", "id": "e0fe3429-dbff-4e83-827a-2e31da60dfc3", "metadata": { "jp-MarkdownHeadingCollapsed": true }, "source": [ "## 数据准备\n", "\n", "获得蛋白质序列及其对应的功能描述是生物信息学中的一个重要任务，通常涉及从公共数据库中检索数据或通过实验方法确定。以下是几种常用的方法和资源，帮助你获取蛋白质序列和功能描述：\n", "\n", "### 1. **使用公共数据库**\n", "\n", "#### a. **UniProt (Universal Protein Resource)**\n", "\n", "- **网址**：[UniProt](https://www.uniprot.org/)\n", "- **特点**：UniProt 是一个综合性的蛋白质数据库，提供了丰富的注释信息，包括蛋白质序列、结构、功能、亚细胞定位等。\n", "- **使用方法**：\n", " - 在搜索栏中输入蛋白质名称、基因名称或序列 ID。\n", " - 浏览结果页面以查看详细的注释信息，包括功能描述、GO（Gene Ontology）术语、文献引用等。\n", "\n", "\n", "\n", "\n", "\n", "\n", "#### b. **NCBI (National Center for Biotechnology Information)**\n", "\n", "- **网址**：[NCBI](https://www.ncbi.nlm.nih.gov/)\n", "- **特点**：NCBI 提供多个相关数据库，如 GenBank、RefSeq 和 Protein 数据库，涵盖广泛的生物物种和蛋白质信息。\n", "- **使用方法**：\n", " - 使用 NCBI 的搜索工具 Entrez 或 BLAST 搜索蛋白质序列或功能描述。\n", " - 访问特定的蛋白质条目以获取详细信息，包括序列、功能、参考文献等。\n", "\n", "#### c. **PDB (Protein Data Bank)**\n", "\n", "- **网址**：[PDB](https://www.rcsb.org/)\n", "- **特点**：PDB 主要包含蛋白质的三维结构信息，但也提供相关的功能描述和文献引用。\n", "- **使用方法**：\n", " - 使用 PDB 的搜索功能查找特定蛋白质的结构信息。\n", " - 查看每个条目的详细页面以获取功能描述和其他相关信息。\n", "\n", "#### d. **Ensembl**\n", "\n", "- **网址**：[Ensembl](https://www.ensembl.org/)\n", "- **特点**：Ensembl 提供基因组浏览器和注释信息，涵盖了多种物种的基因和蛋白质数据。\n", "- **使用方法**：\n", " - 使用 Ensembl 的搜索功能查找特定蛋白质或基因。\n", " - 浏览条目页面以获取详细的注释信息，包括功能描述、GO 术语等。\n", "\n", "### 2. **通过生物信息学工具**\n", "\n", "#### a. **BLAST (Basic Local Alignment Search Tool)**\n", "\n", "- **网址**：[BLAST](https://blast.ncbi.nlm.nih.gov/Blast.cgi)\n", "- **特点**：BLAST 是一种常用的比对工具，可以帮助你根据已知的蛋白质序列找到相似的序列，并获取其功能描述。\n", "- **使用方法**：\n", " - 输入你的蛋白质序列。\n", " - 选择适当的数据库（如 NR、Swiss-Prot 等）进行比对。\n", " - 分析比对结果，查看相似序列的功能描述。\n", "\n", "#### b. **InterProScan**\n", "\n", "- **网址**：[InterPro](https://www.ebi.ac.uk/interpro/)\n", "- **特点**：InterProScan 是一种用于识别蛋白质家族、结构域和重要位点的工具，可以提供详细的注释信息。\n", "- **使用方法**：\n", " - 输入你的蛋白质序列。\n", " - 运行 InterProScan 分析，获取功能描述、结构域信息等。\n", "\n", "### 3. **通过文献和出版物**\n", "\n", "#### a. **PubMed**\n", "\n", "- **网址**：[PubMed](https://pubmed.ncbi.nlm.nih.gov/)\n", "- **特点**：PubMed 是一个广泛使用的生物医学文献数据库，提供了大量关于蛋白质功能的研究论文。\n", "- **使用方法**：\n", " - 使用关键词搜索与特定蛋白质相关的研究论文。\n", " - 阅读论文以获取详细的实验数据和功能描述。\n", "\n", "#### b. **Google Scholar**\n", "\n", "- **网址**：[Google Scholar](https://scholar.google.com/)\n", "- **特点**：Google Scholar 是一个学术搜索引擎，涵盖广泛的科学文献。\n", "- **使用方法**：\n", " - 使用关键词搜索与特定蛋白质相关的研究论文。\n", " - 阅读论文以获取详细的实验数据和功能描述。" ] }, { "cell_type": "markdown", "id": "e794b698-ef26-4695-9b77-ff7314210e8b", "metadata": {}, "source": [ "## 整理好的数据\n", "\n", "https://huggingface.co/datasets/PharMolix/MutaDescribe\n", "\n", "\n", "\n", "https://huggingface.co/datasets/jonghyunlee/UniProt_function_text_descriptions?row=2" ] }, { "cell_type": "code", "execution_count": 1, "id": "e6c59f74-877a-4a74-9017-e61eb713e285", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"\\nimport os\\n\\n# 设置环境变量, autodl专区 其他idc\\nos.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'\\n\\n# 打印环境变量以确认设置成功\\nprint(os.environ.get('HF_ENDPOINT'))\\n\"" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import subprocess\n", "import os\n", "# 设置环境变量, autodl一般区域\n", "result = subprocess.run('bash -c \"source /etc/network_turbo && env | grep proxy\"', shell=True, capture_output=True, text=True)\n", "output = result.stdout\n", "for line in output.splitlines():\n", " if '=' in line:\n", " var, value = line.split('=', 1)\n", " os.environ[var] = value\n", "\n", "\"\"\"\n", "import os\n", "\n", "# 设置环境变量, autodl专区 其他idc\n", "os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'\n", "\n", "# 打印环境变量以确认设置成功\n", "print(os.environ.get('HF_ENDPOINT'))\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": 2, "id": "f038b5f3-b2a5-45bd-b66a-0475b1f2c026", "metadata": {}, "outputs": [], "source": [ "from transformers import AutoTokenizer, AutoModel\n", "from tokenizers import Tokenizer\n", "from transformers import GPT2LMHeadModel, AutoConfig,GPT2Tokenizer\n", "from transformers import AutoModelForSeq2SeqLM \n", "from transformers import DataCollatorWithPadding" ] }, { "cell_type": "code", "execution_count": 3, "id": "7c861666-010e-46d6-aaf0-c63e52920d99", "metadata": {}, "outputs": [], "source": [ "#set tokenizer,dna protein \n", "tokenizer = GPT2Tokenizer.from_pretrained(\"dnagpt/gene_eng_gpt2_v0\")\n", "tokenizer.pad_token = tokenizer.eos_token" ] }, { "cell_type": "code", "execution_count": 4, "id": "32532c2d-962d-4aa0-a823-87d6c62a411f", "metadata": {}, "outputs": [], "source": [ "from datasets import load_dataset\n", "# 1. load ~11k samples from promoters prediction dataset\n", "dataset = load_dataset(\"jonghyunlee/UniProt_function_text_descriptions\")['train'].select(range(5000)).train_test_split(test_size=0.05)" ] }, { "cell_type": "code", "execution_count": 5, "id": "3829dc86-1274-440b-aa2b-e100f112e9bf", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "DatasetDict({\n", " train: Dataset({\n", " features: ['entry', 'entry_name', 'protein_name', 'sequence', 'function'],\n", " num_rows: 4750\n", " })\n", " test: Dataset({\n", " features: ['entry', 'entry_name', 'protein_name', 'sequence', 'function'],\n", " num_rows: 250\n", " })\n", "})" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dataset" ] }, { "cell_type": "code", "execution_count": 6, "id": "6cb4a847-8fd1-4cdd-a905-e0595250c712", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'entry': 'A1TFU9',\n", " 'entry_name': 'HPXO_MYCVP',\n", " 'protein_name': 'FAD-dependent urate hydroxylase (EC 1.14.13.113) (Flavoprotein urate hydroxylase)',\n", " 'sequence': 'MKVVIVGAGMGGMSAAIALRQIGIDTVVYERVTENKPVGAAISVWSNGVKCLNYLGLQEETAELGGKVETMSYVDGHTGDTMCRFSMHPLIEQVGQRPYPIARAELQLMLMKAYGIDDINFGMKMVGVENDTAGSAAKATFADGTTVSADVIIGADGAGSITREYVLGGPVSRRYAGYVNYNGLVSTDDAIGPATEWTTYVGDGKRVSVMPVSDDRFYFFFDVVEPQGSPYEEGRVREVLRAHFAGWTPGVQTLIDTLDPLATNRVEILDLDPFHTWVKGRVAVLGDAAHNTTPDIGQGGCSAMEDAIALQWAFKDHPDDVHAALAAYQSARTERAADLVLRARKRCDVTHAKDPQVTSRWYDELRNEDGTNIIRGIVGNIVGGPLTPVTAATEG',\n", " 'function': 'Catalyzes the hydroxylation of urate to 5-hydroxyisourate (HIU). Is likely to be involved in the urate degradation pathway to allantoin. Prefers NADH over NADPH as the electron donor. '}" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dataset[\"train\"][0]" ] }, { "cell_type": "code", "execution_count": 10, "id": "2eb330e7-28ea-46f1-b9bb-093352e1c5d8", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dna datasets mean token lenght 269.515 min token length 24 max token length 4577\n" ] } ], "source": [ "token_len_list = []\n", "for item in dataset[\"test\"].select(range(200)):\n", " inputs = tokenizer.tokenize(item[\"sequence\"])\n", " token_len_list.append( len(inputs) )\n", "\n", "mean_len = sum(token_len_list)/len(token_len_list)\n", "min_len = min(token_len_list)\n", "max_len = max(token_len_list)\n", "\n", "print(\"dna datasets \", \"mean token lenght\", mean_len, \"min token length\", min_len, \"max token length\", max_len)" ] }, { "cell_type": "code", "execution_count": 11, "id": "f904a4ad-6fe1-4588-b5fd-5541e26a9bfd", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dna datasets mean token lenght 271.02 min token length 23 max token length 1934\n" ] } ], "source": [ "token_len_list = []\n", "for item in dataset[\"test\"].select(range(50)):\n", " inputs = tokenizer.tokenize(item[\"function\"])\n", " token_len_list.append( len(inputs) )\n", "\n", "mean_len = sum(token_len_list)/len(token_len_list)\n", "min_len = min(token_len_list)\n", "max_len = max(token_len_list)\n", "\n", "print(\"dna datasets \", \"mean token lenght\", mean_len, \"min token length\", min_len, \"max token length\", max_len)" ] }, { "cell_type": "code", "execution_count": 6, "id": "73ea434d-6262-4d47-a6e9-3ffb136d8cd0", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "32b5e6b926f24df282da36e7f14cc9ba", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Map: 0%| | 0/4750 [00:00, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::logic_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::~locale()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::basic_string(std::string const&, unsigned long, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_end_catch@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_ofstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::logic_error::~logic_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for __cxxabiv1::__si_class_type_info@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios >::_M_cache_locale(std::locale const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_stringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator new[](unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_M_leak_hard()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ifstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf >::basic_streambuf(std::basic_streambuf > const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::append(char const*, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::basic_string(std::string const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned short@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::resize(unsigned long, char)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for char const*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ctype::_M_widen_init() const@GLIBCXX_3.4.11'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_invalid_argument(char const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::operator=(std::locale const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios >::_M_cache_locale(std::locale const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_decrement(std::_Rb_tree_node_base const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_free_exception@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::condition_variable::notify_one()@GLIBCXX_3.4.11'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::Init::~Init()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::~basic_string()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_pure_virtual@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::flush()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for __cxxabiv1::__class_type_info@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_rethrow@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringbuf, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_fstream >::~basic_fstream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::compare(char const*) const@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_ostringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::locale()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::chrono::_V2::system_clock::now()@GLIBCXX_3.4.19'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_ifstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Hash_bytes(void const*, unsigned long, unsigned long)@CXXABI_1.3.5'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(long long)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for char*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_Prime_rehash_policy::_M_need_rehash(unsigned long, unsigned long, unsigned long) const@GLIBCXX_3.4.18'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::out_of_range@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(unsigned long)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_increment(std::_Rb_tree_node_base const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::~ios_base()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::range_error::~range_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__basic_file::~__basic_file()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_guard_acquire@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(bool)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::overflow_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_fstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::range_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ios >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_filebuf >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator delete[](void*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::basic_string(unsigned long, char, std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_List_node_base::_M_transfer(std::__detail::_List_node_base*, std::__detail::_List_node_base*)@GLIBCXX_3.4.15'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::replace(unsigned long, unsigned long, char const*, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for std::exception@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::_Rep::_M_destroy(std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::istream& std::istream::_M_extract(double&)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf >::close()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_fstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ifstream >::basic_ifstream(char const*, std::_Ios_Openmode)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::append(std::string const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator new(unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_istringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned int@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::append(char const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::domain_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::find(char, unsigned long) const@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::put(char)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for int@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_bad_alloc()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_thread_atexit@CXXABI_1.3.7'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_increment(std::_Rb_tree_node_base*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ifstream >::~basic_ifstream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::Init::Init()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::condition_variable::condition_variable()@GLIBCXX_3.4.11'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf >::basic_filebuf()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_istringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::domain_error::~domain_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::cerr@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::find(char const*, unsigned long, unsigned long) const@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_istringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::basic_string(std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringbuf, std::allocator >::str() const@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::invalid_argument@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for void*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::assign(std::string const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ostringstream, std::allocator >::~basic_ostringstream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned long@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*)@GLIBCXX_3.4.15'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_List_node_base::_M_unhook()@GLIBCXX_3.4.15'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ostringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringbuf, std::allocator >::_M_sync(char*, unsigned long, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_iostream >::~basic_iostream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::locale(std::locale const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_istringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `log2f@GLIBC_2.2.5'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::operator<<(std::basic_streambuf >*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_streambuf >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::exception::~exception()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__basic_file::is_open() const@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_istringstream, std::allocator >::~basic_istringstream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::swap(std::string&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned long*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ostringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf >::basic_streambuf(std::basic_streambuf > const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios >::init(std::basic_streambuf >*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_bad_cast()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios >::clear(std::_Ios_Iostate)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf >::operator=(std::basic_streambuf > const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator delete(void*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::operator<<(int)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_Rep::_S_empty_rep_storage@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_Rep::_M_destroy(std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_iostream >::~basic_iostream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::runtime_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ofstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringstream, std::allocator >::~basic_stringstream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_stringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(long)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::istream::get()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned long long@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ostream >& std::operator<< >(std::basic_ostream >&, char const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::out_of_range::~out_of_range()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::length_error::~length_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ostream >& std::__ostream_insert >(std::basic_ostream >&, char const*, long)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::invalid_argument::~invalid_argument()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::swap(std::basic_string, std::allocator >&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::cout@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(unsigned long long)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for int*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(void const*)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::underflow_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_streambuf >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for std::out_of_range@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_allocate_exception@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ios >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for void const*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios >::init(std::basic_streambuf >*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::reserve(unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_begin_catch@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for long@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::_Rep::_S_empty_rep_storage@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_M_leak()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf >::open(char const*, std::_Ios_Openmode)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringbuf, std::allocator >::_M_sync(wchar_t*, unsigned long, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::istream::getline(char*, long, char)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_istream >& std::getline, std::allocator >(std::basic_istream >&, std::basic_string, std::allocator >&, char)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::condition_variable::~condition_variable()@GLIBCXX_3.4.11'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringbuf, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::insert(unsigned long, char const*, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::assign(char const*, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned char@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::ios_base()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_out_of_range(char const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::overflow_error::~overflow_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_length_error(char const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_system_error(int)@GLIBCXX_3.4.11'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ofstream >::close()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(double)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf >::operator=(std::basic_streambuf > const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for long long@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::basic_string(char const*, unsigned long, std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ifstream >::close()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_guard_release@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_throw@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::underflow_error::~underflow_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_decrement(std::_Rb_tree_node_base*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::length_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf >::~basic_filebuf()@GLIBCXX_3.4'\n", "collect2: error: ld returned 1 exit status\n" ] }, { "data": { "text/html": [ "\n", "
\n", " \n", " \n", " [2970/2970 03:02, Epoch 5/5]\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
EpochTraining LossValidation Loss
15.1236004.953184
24.9586004.907465
34.8935004.876421
44.8010004.860046
54.7096004.876781

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "TrainOutput(global_step=2970, training_loss=4.859284495344066, metrics={'train_runtime': 183.1426, 'train_samples_per_second': 129.68, 'train_steps_per_second': 16.217, 'total_flos': 1551421440000000.0, 'train_loss': 4.859284495344066, 'epoch': 5.0})" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from transformers import Trainer, TrainingArguments, GPT2LMHeadModel\n", "from transformers import DataCollatorForSeq2Seq\n", "\n", "\n", "# 加载预训练的 GPT-2 模型\n", "model = GPT2LMHeadModel.from_pretrained(\"dnagpt/gene_eng_gpt2_v0\")\n", "\n", "data_collator = DataCollatorForSeq2Seq(tokenizer, model=model)\n", "\n", "# 训练参数\n", "training_args = TrainingArguments(\n", " output_dir=\"gpt2_protein_function\", # 输出目录\n", " evaluation_strategy=\"epoch\",\n", " learning_rate=5e-5,\n", " per_device_train_batch_size=8,\n", " per_device_eval_batch_size=8,\n", " num_train_epochs=5,\n", " save_steps=500,\n", " save_total_limit=2,\n", " logging_steps=500,\n", ")\n", "\n", "# 初始化 Trainer\n", "trainer = Trainer(\n", " model=model,\n", " args=training_args,\n", " train_dataset=tokenized_datasets['train'], # 传递 'train' 拆分数据\n", " eval_dataset=tokenized_datasets['test'], # 传递 'test' 拆分数据\n", " data_collator=data_collator,\n", ")\n", "\n", "# 训练模型\n", "trainer.train()\n" ] }, { "cell_type": "code", "execution_count": null, "id": "0e9e5526-39ab-4a75-a09d-f4d14a691211", "metadata": {}, "outputs": [], "source": [ "# 保存训练好的模型和 tokenizer\n", "model.save_pretrained(run_path)\n", "tokenizer.save_pretrained(run_path)\n" ] }, { "cell_type": "code", "execution_count": 19, "id": "18cba087-6620-4d08-ae3a-0ffbd37c7f69", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'eval_loss': 4.8767805099487305, 'eval_runtime': 0.489, 'eval_samples_per_second': 511.252, 'eval_steps_per_second': 65.44, 'epoch': 5.0}\n", "Perplexity: 131.21\n" ] } ], "source": [ "import math\n", "# 评估模型在验证集上的表现\n", "eval_results = trainer.evaluate()\n", "print(eval_results)\n", "print(f\"Perplexity: {math.exp(eval_results['eval_loss']):.2f}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "4e032392-f018-44de-b1f4-0143025a660c", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }