900字范文,内容丰富有趣,生活中的好帮手!
900字范文 > 【修改conda环境和缓存默认路径】修改Python 的 pip install 默认安装依赖路径

【修改conda环境和缓存默认路径】修改Python 的 pip install 默认安装依赖路径

时间:2024-02-26 18:30:52

相关推荐

【修改conda环境和缓存默认路径】修改Python 的 pip install 默认安装依赖路径

文章目录

一、 默认安装路径 1.在本地环境下2.在虚拟环境下 二、修改安装路径 问题:解决: 1、在本地环境下 (1)使用 setup.py安装(2)使用 whl格式文件安装(3)使用 pip、conda安装 2、在虚拟环境下 (1)修改conda环境和缓存默认路径 快捷方式: (2)修改pip默认安装依赖路径 查看包的搜索路径修改用户环境目录

一、 默认安装路径

无论是在操作系统的本地基础环境下,还是在Anaconda或者python的虚拟环境下,一般情况下,我们使用setup.py 、conda或者pip等方式安装一个python的扩展包时,都是安装到当前环境中的python的默认对应路径

例如:

1.在本地环境下

$ which python$HOME/software/python/bin/python12

默认包的安装路径很可能为(对于2.7版本):

$HOME/software/python/lib/python2.7/site-packages1

2.在虚拟环境下

(stone) root@348a24e5dea7:~# which python/root/anaconda3/envs/stone/bin/python12

默认包的安装路径为:

/root/anaconda3/envs/stone/lib/python3.9/site-packages1

二、修改安装路径

问题:

当我们在安装一个python的扩展包时,如果出现以下情况应该怎么解决?

默认的安装路径不是当前环境下python的默认对应路径需要指定安装路径

解决:

1、在本地环境下

(1)使用 setup.py安装

unzip numpy-1.13.1.zipcd numpu-1.13.1#指定安装路径python setup.py install --prefix=/path/to/install # 等号后面写安装路径1234

(2)使用 whl格式文件安装

pip install 123.whl # 123.whl 是whl包的名字#我们可以通过添加 --target=/path/to/install 来指定安装路径12

(3)使用 pip、conda安装

如何修改默认的安装路径,让我们不需要指定例如 --prefix 参数就可以实现呢?

我们可以在$HOME目录下创建一个配置文件 “.pydistutils.cfg” ,用来特别指定python的packages的默认安装目录,例如:

[install]install_lib = $HOME/software/lib/pythoninstall_scripts = $HOME/software/python/bin123

2、在虚拟环境下

(1)修改conda环境和缓存默认路径

默认情况下,conda 创建的新环境 以及过往安装的模块缓存都存储在用户目录下,这一点不会在 conda (user-specific)配置文件 $HOME/.condarc 中体现出来,除非用户修改了默认设置。当前的默认信息可通过 conda info 指令打印查看,其中不仅能够查看当前默认环境路径和默认缓存路径,还能查看conda的源设置。

## 示例(tensorflow) C:\Users\xxx>conda infoCurrent conda install:

platform <span class="token punctuation">:</span> win<span class="token operator">-</span><span class="token number">32</span>conda version <span class="token punctuation">:</span> <span class="token number">4.3</span><span class="token number">.30</span>conda <span class="token keyword">is</span> private <span class="token punctuation">:</span> <span class="token boolean">False</span>conda<span class="token operator">-</span>env version <span class="token punctuation">:</span> <span class="token number">4.3</span><span class="token number">.30</span>conda<span class="token operator">-</span>build version <span class="token punctuation">:</span> <span class="token keyword">not</span> installedpython version <span class="token punctuation">:</span> <span class="token number">3.6</span><span class="token number">.3</span><span class="token punctuation">.</span>final<span class="token punctuation">.</span><span class="token number">0</span>requests version <span class="token punctuation">:</span> <span class="token number">2.18</span><span class="token number">.4</span>root environment <span class="token punctuation">:</span> C<span class="token punctuation">:</span>\ProgramData\Miniconda3 <span class="token punctuation">(</span>read only<span class="token punctuation">)</span>default environment <span class="token punctuation">:</span> C<span class="token punctuation">:</span>\Users\xxx\AppData\Local\conda\conda\envs\tensorflowenvs directories <span class="token punctuation">:</span> C<span class="token punctuation">:</span>\Users\xxx\AppData\Local\conda\conda\envs<span class="token comment">#环境依次存储在下面几个目录,第一个为默认存储目录</span>C<span class="token punctuation">:</span>\ProgramData\Miniconda3\envsC<span class="token punctuation">:</span>\Users\xxx\<span class="token punctuation">.</span>conda\envspackage cache <span class="token punctuation">:</span> C<span class="token punctuation">:</span>\ProgramData\Miniconda3\pkgs<span class="token comment">#缓存依次存储在下面几个目录,第一个为默认存储目录</span>C<span class="token punctuation">:</span>\Users\xxx\AppData\Local\conda\conda\pkgschannel URLs <span class="token punctuation">:</span> https<span class="token punctuation">:</span><span class="token operator">//</span>repo<span class="token punctuation">.</span>continuum<span class="token punctuation">.</span>io<span class="token operator">/</span>pkgs<span class="token operator">/</span>main<span class="token operator">/</span>win<span class="token operator">-</span><span class="token number">32</span><span class="token comment">#源地址</span>https<span class="token punctuation">:</span><span class="token operator">//</span>repo<span class="token punctuation">.</span>continuum<span class="token punctuation">.</span>io<span class="token operator">/</span>pkgs<span class="token operator">/</span>main<span class="token operator">/</span>noarchhttps<span class="token punctuation">:</span><span class="token operator">//</span>repo<span class="token punctuation">.</span>continuum<span class="token punctuation">.</span>io<span class="token operator">/</span>pkgs<span class="token operator">/</span>free<span class="token operator">/</span>win<span class="token operator">-</span><span class="token number">32</span>https<span class="token punctuation">:</span><span class="token operator">//</span>repo<span class="token punctuation">.</span>continuum<span class="token punctuation">.</span>io<span class="token operator">/</span>pkgs<span class="token operator">/</span>free<span class="token operator">/</span>noarchhttps<span class="token punctuation">:</span><span class="token operator">//</span>repo<span class="token punctuation">.</span>continuum<span class="token punctuation">.</span>io<span class="token operator">/</span>pkgs<span class="token operator">/</span>r<span class="token operator">/</span>win<span class="token operator">-</span><span class="token number">32</span>https<span class="token punctuation">:</span><span class="token operator">//</span>repo<span class="token punctuation">.</span>continuum<span class="token punctuation">.</span>io<span class="token operator">/</span>pkgs<span class="token operator">/</span>r<span class="token operator">/</span>noarchhttps<span class="token punctuation">:</span><span class="token operator">//</span>repo<span class="token punctuation">.</span>continuum<span class="token punctuation">.</span>io<span class="token operator">/</span>pkgs<span class="token operator">/</span>pro<span class="token operator">/</span>win<span class="token operator">-</span><span class="token number">32</span>https<span class="token punctuation">:</span><span class="token operator">//</span>repo<span class="token punctuation">.</span>continuum<span class="token punctuation">.</span>io<span class="token operator">/</span>pkgs<span class="token operator">/</span>pro<span class="token operator">/</span>noarchhttps<span class="token punctuation">:</span><span class="token operator">//</span>repo<span class="token punctuation">.</span>continuum<span class="token punctuation">.</span>io<span class="token operator">/</span>pkgs<span class="token operator">/</span>msys2<span class="token operator">/</span>win<span class="token operator">-</span><span class="token number">32</span>https<span class="token punctuation">:</span><span class="token operator">//</span>repo<span class="token punctuation">.</span>continuum<span class="token punctuation">.</span>io<span class="token operator">/</span>pkgs<span class="token operator">/</span>msys2<span class="token operator">/</span>noarchconfig <span class="token builtin">file</span> <span class="token punctuation">:</span> <span class="token boolean">None</span>netrc <span class="token builtin">file</span> <span class="token punctuation">:</span> <span class="token boolean">None</span>offline mode <span class="token punctuation">:</span> <span class="token boolean">False</span>user<span class="token operator">-</span>agent <span class="token punctuation">:</span> conda<span class="token operator">/</span><span class="token number">4.3</span><span class="token number">.30</span> requests<span class="token operator">/</span><span class="token number">2.18</span><span class="token number">.4</span> CPython<span class="token operator">/</span><span class="token number">3.6</span><span class="token number">.3</span> Windows<span class="token operator">/</span><span class="token number">7</span> Windows<span class="token operator">/</span><span class="token number">6.1</span><span class="token number">.7601</span>administrator <span class="token punctuation">:</span> <span class="token boolean">False</span>

12345678910111213141516171819222324252627282930313233

这种默认的处理方式的好处是友好利用了现代操作系统多用户的特点,保证每个用户操作之间的隔离性;问题是在用户个人目录存储空间设置得很小的情况下,随着用户使用时间变长,可用的用户目录空间将越来越少,很可能导致没有足够的空间来容纳新创建的环境或没有足够的空间来为环境安装新的模块。这时候该怎么办?

实际上,conda环境和缓存的默认路径(envs directories 和 package cache)不一定要默认存储在用户目录,我们可以将他们设置到盈余空间稍大的其他目录来缓解这种空间压力,只要保证不同用户之间的设置不同即可。路径的设置可以通过编辑 $HOME/.condarc 来实现。添加或修改 $HOME/.condarc 中的 env_dirs 和 pkgs_dirs 配置项分别设置环境和缓存路径,按顺序第一个路径作为默认存储路径,搜索环境和缓存时按先后顺序在各目录中查找。

// 配置示例channels:- https://mirrors.tuna./anaconda/cloud/conda-forge/- https://mirrors.tuna./anaconda/pkgs/free/- defaultsshow_channel_urls: trueenvs_dirs:- D:\data\xxx\Miniconda\envs- C:\Users\xxx\AppData\Local\conda\conda\envs- C:\Users\xxx\.conda\envs- C:\ProgramData\Anaconda\Miniconda3\envspkgs_dirs:- D:\data\xxx\Miniconda\pkgs- C:\Users\xxx\AppData\Local\conda\conda\pkgs- C:\Users\xxx\.conda- C:\ProgramData\Anaconda\Miniconda3\pkgs

1234567891011121314151617

如上例,新建环境将存储在 D:\data\xxx\Miniconda\envs 下,缓存文件将存储在 D:\data\xxx\Miniconda\pkgs 下,从而避免了用户目录空间不足带来的窘迫。

快捷方式:

conda修改环境安装路径

conda config --add envs_dirs newdir # 增加环境路径conda config --remove envs_dirs newdir # 删除环境路径12

package cache同理,但增加新package cache后会替代默认的两个路径。

修改完后可使用conda info查看。

(2)修改pip默认安装依赖路径

查看包的搜索路径

可以通过 python -m site(类似于 sys.path()的效果) 查看当前环境的包搜索路径(pip默认安装路径)以及用户环境的目录。

这里的USER_BASE 和USER_SITE其实就是用户自定义的启用Python脚本和依赖安装包的基础路径。

其中的USER_SITE其实就是用户如果调用本地基础环境下的python.exe中的脚本pip文件去下载,就会将site-package的默认安装到本地home路径下。

user_site其实就是个人的site-packages默认安装路径了。而如果使用的是Anaconda 目录下的Scripts中的pip 安装依赖库或包等,则是安装在Anaconda路径下的site-packages中去。

总之,pip install 的默认依赖安装路径与当前调用的python位置有关,可通过以下命令确定:

(base) root@348a24e5dea7:~# which python/root/anaconda3/bin/python12

因为 pip install 背后实质上是通过 setup.py 来安装python包的;

而 setup.py 中用到 setuptools,安装过程中会用到sys.path信息,虽然上面sys.path中的信息包含 envs/py38,这个虚拟环境路径;

而且sys.path中的目录排序是有优先级顺序的(envs/py38 路径排在.local后面)

再者,当前用户环境目录也是 .local/, 且会最优先用户环境目录进行安装目录的确认;

修改用户环境目录

注:修改的前提是已经进入虚拟环境,这样修改的是虚拟环境下的配置,而不会影响到全局

修改 site.py 文件:

sudo vim /opt/conda/envs/py38/lib/python3.8/site.py # 是否sudo看情况1

默认 USER_SITE、USER_BASE 都为None (用的时候会默认填充当前用户的)

接下来把他们修改为conda环境的目录:

到此,pip install 的内容成功安装到虚拟环境下。

参看文献:

conda环境的默认pip安装位置

本内容不代表本网观点和政治立场,如有侵犯你的权益请联系我们处理。
网友评论
网友评论仅供其表达个人看法,并不表明网站立场。