Geek asks and answers

Problems compiling Python 3.2 and 2.7 using pythonbrew

I am trying to create multiple versions of python using Pythonbrew , but I am getting some testing failures. This is in the virtual machine: Ubuntu 8.04 32bit

This happens when I use the option --enable-shared

The errors in the build.log file are described below:

Python 3.2

Here are the failed tests for Python 3.2:

LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-3.2: ./python -E -c 'import sys ; from sysconfig import get_platform ; print(get_platform()+"-"+sys.version[0:3])' >platform
find ./Lib -name '*.py[co]' -print | xargs rm -f
LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-3.2: ./python -Wd -E -bb  ./Lib/test/regrtest.py -l 
== CPython 3.2 (r32:88445, Jun 16 2011, 15:07:10) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)]
==   Linux-2.6.18.8-linode22-i686-with-debian-lenny-sid little-endian
==   /opt/standingcloud/stack/pythonbrew/build/Python-3.2/build/test_python_18839
Testing with flags: sys.flags(debug=0, division_warning=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, verbose=0, bytes_warning=2, quiet=0)

....SNIP....

[197/349] test_os
test test_os failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-3.2/Lib/test/test_os.py", line 646, in test_exist_ok_existing_directory
    os.makedirs(path, mode=mode, exist_ok=True)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-3.2/Lib/os.py", line 152, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '@test_18839_tmp/dir1'

....SNIP....

321 tests OK.
1 test failed:
    test_os
27 tests skipped:
    test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp
    test_codecmaps_kr test_codecmaps_tw test_curses test_dbm_gnu
    test_dbm_ndbm test_gdb test_ioctl test_kqueue test_ossaudiodev
    test_pep277 test_smtpnet test_socketserver test_sqlite
    test_startfile test_timeout test_tk test_ttk_guionly
    test_unicode_file test_urllib2net test_urllibnet test_winreg
    test_winsound test_xmlrpc_net test_zipfile64
5 skips unexpected on linux2:
    test_dbm_gnu test_dbm_ndbm test_ioctl test_tk test_ttk_guionly
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='ANSI_X3.4-1968'>
make: *** [test] Error 1

Python 2.7.1

Here are the failed tests for Python 2.7.1:

./python -Wd -3 -E -tt  ./Lib/test/regrtest.py -l 
== CPython 2.7.1 (r271:86832, Jun 16 2011, 17:59:28) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)]
==   Linux-2.6.18.8-linode22-i686-with-debian-lenny-sid little-endian
==   /opt/standingcloud/stack/pythonbrew/build/Python-2.7.1/build/test_python_19286

....SNIP....

test test_file2k failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7.1/Lib/test/test_file2k.py", line 219, in testStdin
    self.assertRaises(IOError, sys.stdin.seek, -1)
AssertionError: IOError not raised

....SNIP....

343 tests OK.
2 tests failed:
    test_file test_file2k
41 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dbm test_gdb test_gdbm test_gl test_imgfile test_ioctl
    test_kqueue test_linuxaudiodev test_macos test_macostools
    test_ossaudiodev test_pep277 test_scriptpackages test_smtpnet
    test_socketserver test_sqlite test_startfile test_sunaudiodev
    test_timeout test_tk test_ttk_guionly test_unicode_file
    test_urllib2net test_urllibnet test_winreg test_winsound
    test_zipfile64
8 skips unexpected on linux2:
    test_bsddb test_bsddb3 test_dbm test_gdb test_gdbm test_ioctl
    test_tk test_ttk_guionly
make: *** [test] Error 1

Python 2.7

Here are the failed tests for Python 2.7:

LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-2.7: ./python -E -c 'import sys ; from sysconfig import get_platform ; print get_platform()+"-"+sys.version[0:3]' >platform
find ./Lib -name '*.py[co]' -print | xargs rm -f
LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-2.7: ./python -Wd -3 -E -tt ./Lib/test/regrtest.py -l 
== CPython 2.7 (r27:82500, Jun 16 2011, 15:25:09) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)]
==   Linux-2.6.18.8-linode22-i686-with-debian-lenny-sid little-endian
==   /opt/standingcloud/stack/pythonbrew/build/Python-2.7/build/test_python_13541

....SNIP....

test_distutils
/usr/bin/ld: cannot find -lpython2.7
collect2: ld returned 1 exit status
test test_distutils failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/tests/test_build_ext.py", line 269, in test_get_outputs
    cmd.run()
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/command/build_ext.py", line 449, in build_extensions
    self.build_extension(ext)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/command/build_ext.py", line 531, in build_extension
    target_lang=language)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/ccompiler.py", line 741, in link_shared_object
    extra_preargs, extra_postargs, build_temp, target_lang)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/unixccompiler.py", line 258, in link
    raise LinkError, msg
LinkError: command 'gcc' failed with exit status 1

....SNIP....

test test_file failed -- multiple errors occurred; run in verbose mode for details
test_file2k
test test_file2k failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/test/test_file2k.py", line 211, in testStdin
    self.assertRaises(IOError, sys.stdin.seek, -1)
AssertionError: IOError not raised

....SNIP....

339 tests OK.
3 tests failed:
    test_distutils test_file test_file2k
1 test altered the execution environment:
    test_site
41 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dbm test_gdb test_gdbm test_gl test_imgfile test_ioctl
    test_kqueue test_linuxaudiodev test_macos test_macostools
    test_ossaudiodev test_pep277 test_scriptpackages test_smtpnet
    test_socketserver test_sqlite test_startfile test_sunaudiodev
    test_timeout test_tk test_ttk_guionly test_unicode_file
    test_urllib2net test_urllibnet test_winreg test_winsound
    test_zipfile64
8 skips unexpected on linux2:
    test_bsddb test_bsddb3 test_dbm test_gdb test_gdbm test_ioctl
    test_tk test_ttk_guionly
make: [test] Error 1 (ignored)
+3
source share
3 answers

I am currently managing to get pythonbrew to work with 2.7.2 by doing the following:

a) First install all the dependencies needed to compile python.

'curl' # not for build, but for steps after
'build-essential',
'libbz2-dev',
'libsqlite3-dev',
'zlib1g-dev',
'libxml2-dev',
'libxslt1-dev',
'libreadline5', # lenny
'libreadline5-dev', # lenny
'libgdbm-dev',
'libgdb-dev',
'libxml2',
'libssl-dev',
'tk-dev',
'libgdbm-dev',
'libexpat1-dev',
'libncursesw5-dev'

b) Now install 2.7.2:

pythonbrew install --configure="--with-threads --enable-shared" -j2 -v 2.7.2

, :

/home/python-deploy/.pythonbrew/pythons/Python-2.7.2/bin/python: : libpython2.7.so.1.0: shared object file:

* setuptools, -no-setuptools, -force , gdb - .

pythonbrew install --configure="--with-threads--enable-shared" \
                   --force \
                   --no-setuptools \
                   --jobs=2 \
                   --verbose 2.7.2

c) - ( pythonbrew):

pythonbrew use 2.7.2
export LD_LIBRARY_PATH=$HOME/.pythonbrew/pythons/Python-2.7.2/lib

, pythonbrew lib , /etc/ld.so.conf.d, ldconfig:

sudo echo /home/user/.pythonbrew/pythons/Python-2.7.2/lib >> /etc/ld.so.conf.d/pythonbrew.conf
sudo ldconfig

d) setuptools , ..

curl -O http://python-distribute.org/distribute_setup.py
python distribute_setup.py && easy_install pip

. 2.7.2 debian lenny VM.

+4

, LD_LIBRARY_PATH.

, , LD_RUN_PATH, LD_LIBRARY_PATH, , , , ( LD_LIBRARY_PATH ).

, LD_RUN_PATH pythonbrew Python, , . , , distutils pip, - .

LD_RUN_PATH=$HOME/.pythonbrew/pythons/Python-2.7.3/lib pythonbrew install -j2 -C '--enable-shared' --force -v 2.7.3

pythonbrew use 2.7.3, , , init - .

, , pythonbrew Python, LD_RUN_PATH. , $HOME/.pythonbrew/pythons/Python-$VERSION/lib, , , ( --enable-shared), lib.

, , --enable-shared Python Python , , - Pythons. LD_LIBRARY_PATH LD_RUN_PATH Python .

+4

... LD_RUN_PATH - .

:

LD_RUN_PATH='$ORIGIN/../lib'

... $ORIGIN - PWD (, python).

# objdump -x /home/arisinger/.pythonbrew/pythons/Python-2.7.3/bin/python|grep RPATH
  RPATH                $ORIGIN/../lib
# ldd .pythonbrew/pythons/Python-2.7.3/bin/python
    [...]
    libpython2.7.so.1.0 => /home/arisinger/.pythonbrew/pythons/Python-2.7.3/bin/../lib/libpython2.7.so.1.0 (0x00007f43994f6000)
    [...]

... pip .. , man ld.so:

$ORIGIN and rpath
    ld.so  understands the string $ORIGIN (or equivalently ${ORIGIN}) in an
    rpath specification (DT_RPATH or DT_RUNPATH) to mean the directory con‐
    taining  the  application  executable.  Thus, an application located in
    somedir/app could be compiled with gcc  -Wl,-rpath,'$ORIGIN/../lib'  so
    that  it  finds  an  associated shared library in somedir/lib no matter
    where somedir is located in the directory hierarchy.  This  facilitates
    the  creation  of  "turn-key"  applications  that  do  not  need  to be
    installed into special directories, but can instead  be  unpacked  into
    any directory and still find their own shared libraries.

: venvs , virtualenv python ( $ORIGIN), ( ) , C/ venv end up link to system python... ungood:-(...

... IMO is a bug that needs to be fixed (in virtualenv?), But the easy workaround is to manually copy / symlink libpythonX.Y.so*to lib / dir venv ... or to do something a little yucky (XYZ note):

LD_RUN_PATH='$ORIGIN/../lib:$ORIGIN/../../../../pythons/Python-X.Y.Z/lib'

... select anything that is less disgusting; both tested for work.

+3
source

More articles:


All Articles