Matplotlib for Python scripting in ArcGIS 10 does not work

[werner.rolf]

I use ArcGIS 10 and would like to run Matplotlib to create charts within a python script. Therefore I have installed
- numpy-1.6.1rc1-win32-superpack-python2.6.exe
- matplotlib-1.0.1.win32-py2.6.exe

as noted here: http://matplotlib.sourceforge.net/users/installing.html

If I test a script to check the installation like:

import numpy
arcpy.AddMessage("NUMPY Version " + numpy.__version__)

import matplotlib
arcpy.AddMessage("MATPLOTLIB Version " + matplotlib.__version__)

I get the following error:
Running script Script...
NUMPY Version 1.6.1rc1
<type 'exceptions.AttributeError'>: 'module' object has no attribute '__version__'
Failed to execute (Script).

Also the command
- import pylab as pl

leads to the error
<type 'exceptions.ImportError'>: No module named pylab
Failed to execute (Script).


So even matplotlib is installed properly in the directory "site-packages" (see screenshot of directory structure attached) it does not work in ArcGIS.


By the way if I let it run in iPython (same installation) it works perfect.

What still needs to be done to get matplotlib work in ArcGIS 10?

Thanks
Werner

[vsfoote]

I use ArcGIS 10 and would like to run Matplotlib to create charts within a python script. Therefore I have installed
- numpy-1.6.1rc1-win32-superpack-python2.6.exe
- matplotlib-1.0.1.win32-py2.6.exe

Werner,

1) Looking at your install you didn't remove numpy-1.3, supposedly there are ArcPy DCOM elements hard coded against numpy-1.3, but having both releases present is probably not good.

Also, looking at the matplotlib site, only numpy-1.1 is required--did you need numpy-1.6 for some reason? Perhaps the numpy-1.3 will suffice for your plotting work and simplify your install.

The matplotlib Windows 32-bit installer at SourceForge seems to correctly install with the ESRI instance of Python 2.6.

That should get the matplotlib functional for you leaving your ESRI ArcGIS10.0 Python instance intact. I was able to do just this now including the PyReadline and IPython installers. All seem to work OK with just numpy-1.3.

If still having problems, you might want to look for any environment variables or .pth files that may need adjustment to the ESRI default Python installation at C:\Python26\ArcGIS10.0 that may be causing problems.



2) Alternatively, you may be able to load the Python 2.6 build of Python(X,Y)--uses numpy-1.6.0, and again there remains any dependencies of ArcPy on having a numpy-1.3 egg in place.

You would first need to uninstall the ESRI Python instance from Uninstall or Change a Program--saving a copy of the Desktop10.pth file from the site-packages directory to reuse in the new installation. You'll then have to test what breaks in ArcPy without numpy-1.3.

Good luck.

Stuart

[werner.rolf]

Thanks first!
I wasn't aware that there were two versions of numpy running and deinstalled 1.6 again.

I now went completely through your 1st proposal:
So now I now have numpy 1.3 and the matplotlib Windows 32-bit installer that you recommended on my machine. Unfortunatly it still does not work. What confuses me is that numpy seem to work, it's more that matplotlib seem to have the problem i.e. pyplot that comes along with it.
I always get the warning: No module named pyplot
I also checked the pth file and have three variables in there:
C:\Programme\ArcGIS\Desktop10.0\bin
C:\Programme\ArcGIS\Desktop10.0\arcpy
C:\Programme\ArcGIS\Desktop10.0\ArcToolbox\Scripts

Do I need other in there?

At least so far this seem not to work for some reason.

Before I will try your second proposal I'm still looking for alternatives as I do have one of those campus licences that I'm only able to install at university and if I screw this up here I'll be in big mess as I cannot go there in next time and have to get my work done before that.

[vsfoote]

Werner,

Any chance you just need to issue the call to pyplot as matplotlib.pyplot?

Those entries in your Desktop10.pth file look OK.

Your %PATH% variable should have a C:\Python26\ArcGIS10.0 entry, and python.exe and pythonw.exe should only be in that directory.

Also, have a look in the registry at this Key:

HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.6\PythonPath
Note: would be SOFTWARE\Wow6432Node\Python on 64-bit OS.

The key's Default value should have a lib-tk value laid down by the matplotlib installer that matches ESRI's Python 2.6 instance. Something like this:

C:\Python26\ArcGIS10.0\Lib;C:\Python26\ArcGIS10.0\DLLs;C:\Python26\ArcGIS10.0\Lib\lib-tk

If it was not laid down in the same path, you'll need to uninstall matplotlib, and numpy packages. Then reinstall numpy-1.3.0 and matplotlib-1.0.1.
When the package installer GUIs run be sure to choose the C:\Python26\ArcGIS10.0 instance of Python.

Stuart

[werner.rolf]

The registry key is fine.

Sorry but I didn't get what you meant if I need to issue the call to pyplot as matplotlib.pyplot.
Furthermore where do I have to look if the %PATH% variable has a C:\Python26\ArcGIS10.0 entry?

What may also confuse is the fact, that if I run IPython any scripts with arcpy and matplotlib work totally fine, but not if I run the same script in ArcGIS or IDLE that comes along with its installation.

Thanks for your patience!

[vsfoote]

Werner,

Furthermore where do I have to look if the %PATH% variable has a C:\Python26\ArcGIS10.0 entry?

From a command prompt issue:

echo %PATH%

From a regedit session browse to:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment

Or probably best (because you can see both System and Current User values) select:

Start --> Computer --> Properties (System Properties) --> Advanced System Settings (Advanced tab) --> Environment Variables

The Current User will have a Path -- just be sure if it has any Python related paths they do not conflict with the C:\Python26\ArcGIS10.0 location. And in the System Variables block that in the PATH listing, that C:\Python26\ArcGIS10.0 is the only listed Python--or at the least it must be the first listed.

Here is a sample pytplot script that should run displaying a Tcl/Tk plot from IDLE, PyWin32, IPython. It should run even in the ArcMap Python window (either enter each line or load it as a script).

import numpy as np
import matplotlib.pyplot as plt

x = np.arange(0, 5, 0.1);
y = np.sin(x)
plt.plot(x, y)

plt.show()

If you can get it to run in IDLE and the ArcMap Python window you've fixed any path or configuration issues.

Syntax of the packages in your code is another issue.

Stuart

[werner.rolf]

I have checked the path but need not to change anything.
The Path looked like:
C:\Python26\ArcGIS10.0\Lib\site-packages\PyQt4;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;

Your script example works at least in IDLE - for what kind of ever reason as others do not. But if I tart it within ArcMap I get the error:
<class '_tkinter.TclError'>: bad screen distance "640.0"
Failed to execute

Im wondering to give it up or suggest this questions to ask at the next ESRI User Conference.

Furthermore there seem to be a need of some serious improvements on the python interface of ArcGIS, as other matplotlib scripts that are working with python regularly don't work if lauchend in ArcMap.
Hopefully in next versions this will work more consistent.

Anyway thank you very much for your help!

[vsfoote]

Werner,

The Path looked like:
C:\Python26\ArcGIS10.0\Lib\site-packages\PyQt4;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;

With that PATH value you do not have an explicit path to the Python program and libraries.

IMHO to run Python from the command line and for proper functioning of many programs, an entry in the PATH allows you explicit control of the Python executable called. If you did have multiple versions of Python installed, i.e. Python(X, Y) as mentioned, you would control the active version by its position in the PATH.

Here is a sample System Variable Path from my Windows 7 64-bit test workstation--graphic device drivers, operating system, then Python with other applications further down the search path.

C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Python26\ArcGIS10.0;C:\Perl64\site\bin;C:\Perl64\bin;C:\Program Files (x86)\QuickTime\QTSystem;c:\HEGtools\HEG_Win\bin;C:\Program Files\AutoCAD Raster Design 2010\;C:\Program Files\MATLAB\R2011a\bin;C:\utils\trid;C:\FlexNet32

With C:\Python26\ArcGIS10.0 included, this allows one to issue a python or pythonw command or import an associated library under that path from any shell enabled application on the system.

I also have a PYTHONPATH System Variable that as an explicit location takes precedence over standard libarary locations or .PTH files--in this instance I've duplicated the ArcGIS Desktop10.pth values.

C:\Program Files (x86)\ArcGIS\Desktop10.0\bin;C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy;C:\Program Files (x86)\ArcGIS\Desktop10.0\ArcToolbox

The problem with the tkinter.TclError -- bad screen distance "640.0" may be related to your numerical locale setting. In German Windows the "," decimal mark causes some issues.

Found this as a work around, maybe it will help with matplotlib in general with ArcGIS.

import locale
locale.setlocale(locale.LC_NUMERIC, 'C')

here:

http://trac-hacks.org/ticket/1872
http://www.mail-archive.com/matplotl.../msg21373.html.
http://old.nabble.com/%22bad-screen-...d31964358.html

Stuart

[werner.rolf]

Thanks once again, the numerical locale setting code works. And in deed its once agtain one step further. I also changed the path now, so that it now looks like:
C:\Python26\ArcGIS10.0;

Nevertheless the error No module named pylab still remains.

[vsfoote]

Well it's progress

I believe that pylab is a module of the matplotlib site-package. Don't you first need to import matplotlib to be able to call a pylab function?

Or you might try the pylab calls explicitly, but that may leave you without some of the other matplotlib functions:

import matplotlib.pylab as pl

Stuart

[passiveoptical]

No sure if this will help at this point, but here are instructions that I put together to install python(x,y). Stuart's little plotting script works fine for me in IDLE, Spyder, and Arc's Python window.



1) Install ArcGis10 without python or uninstall python
2) Download Python(x,y) from http://www.pythonxy.com/ and install, making sure you select "custom directories" so that python is installed in C:\Python26. If installing on a 64 bit machine, do not include GDAL 1.6 – it won’t work. Instead, go to http://www.lfd.uci.edu/~gohlke/pythonlibs/ and install GDAL-1.8.0.win32-py2.6.exe.
3) Create a file called Desktop10.pth, which should contain these lines:
C:\Program Files (x86)\ArcGIS\Desktop10.0\bin
C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy
C:\Program Files (x86)\ArcGIS\Desktop10.0\ArcToolbox\Script
4) Place the file in the python installation directory: ‘C:\Python26’
5) Open IDLE and try ‘import arcpy’

[vsfoote]

Chris,

Yes that is the direction for using an alternate Python instance. The problem remains ArcGIS 10 ArcPy compiled PYC (DCOM against ArcObjects) dependencies on a hard coded numpy-1.3 instance.

While it was his original question, we'll cook Werner's noodle if we drag him down that path.

For now I'd like to see him get a functional matplotlib site-package install that works with both an ArcGIS 10 Python window--in process--, as well as in a Python IDE or command line for doing the plotting and MATLAB like processing of his geospatial data.

The Python(X, Y) install is an open question over on the GIS Stack Exchange and this one. But your note on GDAL 1.6 vs 1.8 version and past comments on numpy-1.3 dependencies of the ArcPy site-package mean there is a lot to be worked out. What GDAL release does ESRI use in ArcPy?

Stuart

[passiveoptical]

Fair enough. Sorry if I'm sowing mass confusion. Interestingly, I'm running numpy version 1.5.1 and have had no issues. (Yet.)

[werner.rolf]

As my noodles are not cooked yet I was as well considering to try it with Python(x,y) - so it's not really confusing but an option. But somewhere else I read the same about that ArcGIS gets problem with numby newer than 1.3.

Maybe it's a try worth...

[werner.rolf]

Chris,

but if I install Python(x,y) don't I have to install it in a directory namend C:\Python26\ArcGIS10.0\
as this is the default when ArcGIS installed python2.6?

Thanks

[passiveoptical]

but if I install Python(x,y) don't I have to install it in a directory namend C:\Python26\ArcGIS10.0\
as this is the default when ArcGIS installed python2.6?

Hi Werner,

Don't install python in C:\Python26\ArcGIS10.0\. The safest way to start off is probably to completely uninstall arc, then reinstall it without python. Then install python(x,y) in ‘C:\Python26’ and deal with the gdal issue, if necessary. From there, I'll direct you to steps 3 and 4 in my 'Python (x,y) installation instructions:'

3) Create a file called Desktop10.pth, which should contain these lines:
C:\Program Files (x86)\ArcGIS\Desktop10.0\bin
C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy
C:\Program Files (x86)\ArcGIS\Desktop10.0\ArcToolbox\Script
4) Place the file in the python installation directory: ‘C:\Python26’

The Desktop10.pth file will point to the required directories.

If there are issues with using a version of numpy other than 1.3, I haven't encountered them, and frankly, the benefits of having libraries like matplotlib and scipy work properly far outweigh the risks when arc is already such an unreliable bit of software.

Obviously I can't provide a warranty with this, but it works for me and I haven't looked back. Esri really needs to reevaluate it's python distribution so that it can play nice with the wider community, otherwise, what's the point?

I apologize for the rant. Good luck!

Chris