Why would I install ArcSDE with 10.1? It seems I can us ArcCatalog to create an enterprise geodatabase without SDE then have all my users use direct connections. What am I missing?
Why would I install ArcSDE with 10.1? It seems I can us ArcCatalog to create an enterprise geodatabase without SDE then have all my users use direct connections. What am I missing?
With ArcGIS 10.1 you can connect to an enterprise database that does not have an SDE geodatabase repository and create feature classes with a native spatial type. For example, you could connect to a SQL Server database and import/create feature classes with the GEOMETRY or GEOGRAPHY spatial type. However, you can only view this feature class. You cannot perform any edits, or have the feature class participate in any geodatabase functionality (such as replication, topology, geometric networks, relationship classes, network datasets, etc). Essentially, you are getting a 'view' only feature class w/o SDE.
Last edited by JSkinn3; 05-11-2012 at 07:25 AM.
To further add on to what Jake said...
Prior to 10.1 (10.0, 9.x, 8.x) if you wanted to use a Geodatabase with all of the cool Geodatabase functionality (versioning, archiving, topology, relationship classes, terrains, geometric networks, etc) on a DBMS (Oracle, SQL Server, Postgresql, Informix, DB2) you had to perform a few steps.
First, you needed to install ArcSDE onto your machine, usually this was your DBMS server. This install included the files needed to run ArcSDE command line utilities, the application server, as well as the ArcSDE post installer.
After installing this onto your machine you would need to run the ArcSDE post installer. The main purpose of this post install was to install the Geodatabase into your enterprise database. This includes all the stored procedures, functions, privileges, and schema needed to provide the functionality I mentioned earlier. The post install could also be used to set up your application server.
The application server can be used to connect from a client machine to the DBMS/Geodatabase. It is used more or less to manage the connection requests coming in from clients and provide a way for the clients to 'talk' to the DBMS. For a while this was the only way to connect to a Geodatabase. At some point (I think 9.0) we added the ability to make direct connections (2-tier: client-DBMS) to the DBMS. This made the use of the application server (3-tier: client-app server-dbms) optional. For some people this meant they stopped setting up the application server and started using direct connections. Others continued using the application server.
The ArcSDE command line utilities are a method for the Geodatabase administrator to manage data, users, the application server service.
Fast forward to 10.1 - We have tried to allow you to manage your Geodatabase completely within ArcGIS applications (ArcCatalog, ArcMap, ArcGIS Server, etc). This is done through the use of dialogues in ArcMap/Catalog and the use of geoprocessing. The first thing we did was to break out the installation of the geodatabase schema tasks into geoprocessing tools. If you want to create an enterprise geodatabase there are now two options. Option 1, you can use the 'Create Enterprise Geodatabase' geoprocessing tool. This tool will create a new empty geodatabase in an existing instance. The second option is to use the 'Enable Enterprise Geodatabase' tool will allow you to install the Geodatabase schema in an already existing instance. The new functionality that Jake mentioned that allows you to now connect to a enterprise database (not a geodatabase) is what allow ArcGIS to then enable geodatabase behavior in your enterprise geodatabase. This second option would be used where you have already set up a database, have user permissions assigned and maybe have loaded some data (essentially converting your database to a geodatabase). The first option would be used if you are starting from nothing.
Esri recommends using direct connections for making connections to your geodatabase, it is not mandatory that you install the application server.
Much of the commonly used functionality found in the ArcSDE command line utilities is now available either through ArcGIS applications mentioned earlier or through geoprocessing (disconnecting users, identifying locks, loading data, investigating data, etc). For most users the install of these utilities should not be necessary.
If you determine that you really need the application server or the command line utilities they are available as a separate install.
So, this has been a pretty long winded answer to a pretty straightforward question. Answer is, it's not mandatory to install the application server or command line utilities. If you want to take advantage of Geodatabase behavior you do need to run one of the geoprocessing tools to install the Geodatabase.
Last edited by russellb; 10-18-2012 at 08:34 AM.
Thank you very much for the education.
at 10.1 there is no need to install the ArcSDE software unless you need to run an ArcSDE service. If all of your users are making Direct Connections to the geodatabase then the ArcSDE installation is not necessary. As well, most of the functionality offered by ArcSDE commands is now available in ArcGIS Desktop & through GP tools.
the answer from russellb is really clear, but I have another little question:
what about editing?
I have some services that I can edit on the web with ArcGIS Server 10.1, should I still use the SDE connection or it is ok to use a direct Connection?
Is the new ArcGIS Server able to 'replace' or acts as the SDE layer?
Direct Connect *is* an ArcSDE connection. This thread is about application server install,
but the ArcSDE functionality still exists -- It's ArcSDE that provides the basis for versioned
geodatabases. With Direct Connect, it's just a multi-threaded solution in the ArcGIS client
instead of a multi-processing solution on the database server.
Is the qouted text true if I don't install ArcSDE 10.1?You cannot perform any edits, or have the feature class participate in any geodatabase functionality (such as replication, topology, geometric networks, relationship classes, network datasets, etc). Essentially, you are getting a 'view' only feature class w/o SDE.
It has nothing to do with whether or not ArcSDE is installed on your database server. As Vince says, even if you don't use or install it on your database server, ArcSDE is still part of your client ArcGIS application, there are by default DLLs installed on your own computer with ArcGIS that handle the geodatabase SQL logic ---> That is essentially what ArcSDE is!
E.g., have a look at your "C:\Program Files (x86)\ArcGIS\Desktop10.X\Bin" folder on your local computer. You will see DLLs like "sde.dll" and "sdesqlsrvr100.dll" etc.
As soon as you attempt to connect to an ESRI Enterprise Geodatabase, these DLLs will be in use by your client ArcGIS application. There is no way around this when connecting to an ESRI geodatabase - at least for full functionality including editing - ArcSDE is just the component ESRI devised to handle the connection to the database and SQL stuff needed to allow advanced geodatabase functionality like versioning.
There is nothing special about ArcSDE or these DLLs in this respect, other vendors like Bentley or Autodesk have similar software components in their software to handle connections and SQL stuff related to geospatial databases, they just call it differently.
Last edited by mboeringa2010; 11-18-2012 at 04:58 AM.
Just to make sure that I understood you correctly.
What you are saying is I do not need to install ArcSDE on the database server because the client contains the SDE functionality.
Another way of sying it:
- I could use direct connection without ArcSDE installed for two-tier topology.
- I could use ArcSDE to hide or conceal the database in a three-tier topology.
am I right?
Also, rephrasing the first sentence to:
- I could use direct connection without ArcSDE installed on a server for two-tier topology.
is probably a bit more accurate, as, as you now understand, ArcSDE is always installed on the client side as part of the ArcGIS installation.
ArcSDE is the engine of the "car" called "geodatabase"... Take away the engine entirely, and you will grind to a halt.
Last edited by mboeringa2010; 11-18-2012 at 09:01 AM.
I'm trying to wrap my head around this, as I setup my test server for our upgrade.
I have set up my server with Windows Server 2008 R2 sp1 and SQL Server 2012 standard edition.
I have installed ArcGIS desktop 10.1 + sp1
I ran through the ArcGIS for Server 10.1 sp1 installer but did not create a ArcGIS Server site. (essentially I needed the authorization file for the create enterprise geodatabase tool )
I then ran the tool > 'Create Enterprise Geodatabase'
I loaded some data into the geodatabase, registered as versioned.
When exploring the schema all the SDE and GDB tables are present.
Am I missing something? Do I need to install ArcSDE 10.1 to get the multi user/versioning/history capabilties of ArcSDE, or as this discussion indicates this is now inherent in the desktop.
I still am not 100% clear why I would need to install the ArcSDE application server or command line tools.
If I am managing data via desktop (ArcMap & ArcCatalog) I do not need to install anything further
If I am managing data via web editing tools I do need to install the ArcSDE application server.
Thank you for any clarification you can provide.
Good, this means you are most likely "up-and-running", and can start using ArcGIS to fill and use your ESRI geodatabase
As I and others have attempted to explain in this thread, ArcSDE's software components in the form of DLLs (Dynamic Link Libraries) form an inherent - and VITAL - part of any "client" ArcGIS application, most prominently "ArcGIS for Desktop" and "ArcGIS for Server" (Server is after all a "client" of the DBMS too, e.g. Oracle, Microsoft SQL Server). They are installed by default as part of the respective software's installation.
You can't access or run an ESRI geodatabase without these DLLs that incorporate all the functionality and logic of ArcSDE to handle ESRI geodatabases (at least not by re-doing years long software development by ESRI and at great risk of corrupting the database in case you want to edit something).
Like I wrote in my last post:
ArcSDE is the "engine" of the "car" called "geodatabase"... Take away the engine entirely, and you will grind to a halt.
However, if this sentence still leaves you confused, think of the ArcSDE application server running on a remote server being "public transport", and the ArcSDE DLLs on your local computer as part of "ArcGIS for Desktop" as your "private car". Both share an "engine" (THIS IS ARCSDE!), but they are independent and both get you from A to B (allow you to access an ESRI geodatabase with all of it's functionality).
It is up to you to decide if you want to travel by "public transport" or use your "private car". One mode of transport may be faster than the other (or the other way around), depending on the conditions in your local "area"...
To also recap some of the very good comments by the ESRI staff in this thread (Russell Brennan in this case):
"Fast forward to 10.1 - We have tried to allow you to manage your Geodatabase completely within ArcGIS applications (ArcCatalog, ArcMap, ArcGIS Server, etc). This is done through the use of dialogues in ArcMap/Catalog and the use of geoprocessing."
"If you determine that you really need the application server or the command line utilities they are available as a separate install."
These comments mean that only in some exceptional cases, you might need one of the command line tools. Until you run into some serious trouble that really can't be dealt with using ArcGIS's new tools, you should be fine using Direct Connect and leaving your setup as it is now.
Last edited by mboeringa2010; 12-22-2012 at 01:53 AM.
HI All, Please help me understand this for the personal ArcSDE. For the previous item...
No, you do not need to install an ArcSDE 10.1 application server to get full geodatabase functionality, including multi user/versioning/history.
So can I still use a personal ArcSDE with Desktop without ArcGIS Server, say on a laptop? If so, do I need to through through the same procedure for creating or enabling a enterprise geodatabase? If, so how. The create geodatabase requires an authorization code for ArcGIS Server.
Additionally, if you want to create an enterprise geodatabase (so no file geodatabase), you will need an ArcGIS for Server commercial licence, as ArcSDE / ArcGIS for Server is a sale / non-free product of ESRI, and using an enterprise geodatabase requires this licence, even if you only start using it for testing purposes. That is why you are being asked for an authorization code.
You may be able to get a temporary trial licence for free for testing purposes, IDK, but as soon as you start deploying it, you will surely need a true paid licence. For that, there are two possible licencing levels: "ArcGIS for Server Enterprise" and "ArcGIS for Server Workgroup", see this page:
Enterprise geodatabases, which run in many different flavors of RDBMS software, require an
ArcGIS for Server Enterprise license. Desktop (formerly known as "Personal") and Workgroup
multiuser geodatabases (which only run in SQL-Server Express, and are subject to restrictions
in user count, storage, RAM, and CPU cores) are available at different licensing levels.
The Multiuser Geodatabase page breaks down the options.
Desktop multi-user geodatabases are to Enterprise ArcSDE as "Physics for Social Science Majors"
is to "Intermediate Electricity and Magnetism" -- the flavor is there, but a good deal of the rigor
I have a similar question, regarding Oracle.
Some of you here said that if I do not install ArcSDE application on the server, and use only ArcGIS for Desktop, I'll need to install an Oracle Client (32bit ofcourse) on each desktop that will direct connect to the geodatabase via ArcCatalog/ArcMap.
However, I need to be able to connect to the geodatabase (direct connection or application connection) via a desktop without an Oracle client installed. For that, I'll need to install the ArcSDE application on the server, correct?
Also, will I need to use application connection or a direct connection?
In this scenario, the connection is called an Application Server connection, so you won't be using Direct Connect.
Please note you need to add the following line to your Windows services file of your client PC running ArcGIS for Desktop per installation instructions for the ArcSDE Application Server. Please note this has to be done on each machine connecting to your Application Server!:
The Windows services file is simply called "services" without a file extension and in Windows 7 located in:
The services file on the application server must be modified because the service
is started by name. It's good practice to distribute a modified services file among
clients, but it's only necessary if those clients will also lookup by name (if you
specify the port number, then it isn't necessary).
This is exactly what I did, works perfectly.
I have a question though - I've heared a rumor that ESRI will probably eventually stop supporting Application Server Connection, is that true?
What is the "recommended" method of connection by ESRI?
Enterprise Geodatabase 101 here on the ESRI website), there isn't a whole lot to win by abandoning the Application Server option.
From a "support" point of view, there may be reasons, as it seems the Application Server option seems to be a bit more difficult to get up-and-running for some users, and causing more questions and confusion requiring ESRI intervention.
As for "recommended", ESRI made Direct Connect default, so I guess this would qualify as the "recommended" option. I doubt though, Vince would give you such an answer, as he probably justly and more precisely will tell you it all depends on the configuration of your specific client/server hard- and software LAN network environment.
As I said before in this thread:
"It is up to you to decide if you want to travel by "public transport" or use your "private car". One mode of transport may be faster than the other (or the other way around), depending on the conditions in your local "area".."
Last edited by mboeringa2010; 01-21-2013 at 02:19 AM.
Also, in terms of Application Server versus Direct Connect, and why ESRI may have decided to make Direct Connect default, it may have to do with a number of (or probably many) clients of ESRI having seen a significant rise in the number of concurrent users using enterprise geodatabases (and not just through cached webservices).
Where in the past for most organizations it used to be that only a few highly active editors / viewers concurrently accessed the database (e.g. maximum a dozen) through ArcMap, now for some shops maybe dozens or even hundreds might access it.
In that scenario, Direct Connect is probably the best option.
However, if you still are within a mid-size organization, with maybe a (few) dozen maximum active users at any point in time, and have a beefy modern 8+ core database server / Application Server with a tens or hundreds of Gigabytes big enterprise geodatabase, than Application Server connections may still be very much a valid choice. In fact, the server with it's specialized high performance hardware and processors (e.g. Xeon), may outperform your desktop in processing the GIS data, especially in cases of network bandwidth constraints.
Also, this remark by D.E.Wright on the GIS StackExchange Forum may be of interest:
“There are some very good reasons to use an ArcSDE Server Engine (Edit: Should be ArcSDE Application Server in official ESRI terminology), the first being the load. When you utilize a ArcSDE (Edit: Application) Server Service you are taking the bulk of that data load off the database server and queuing it versus relying on just your local machine to store all that temp data.
One thing you will see especially with a MSSQL server when you make your initial database connection in a MXD is that ArcGIS does a 'SELECT *' (as seen in your query analyzer and logs on your DB Server) against that table/feature-class. Now, this can be a huge impact if you have very large datasets; the ArcSDE Service/Process helps in this by aiding in the request of the appropriate data scope.
Now as we have all gotten bigger machines, with more RAM its much easier to just load everything into the current session and run with it; but don't just discount the idea of using the service just because the ArcGIS docs say you 'don't need it' anymore, versus when you probably could/should use it.”
Last edited by mboeringa2010; 01-21-2013 at 06:11 AM.
I think the length and complexity of this thread make a VERY good case for someone overhauling BOTH the ArcSDE and ArcGIS Server installation documentation. This represents a significant departure from previous architectures and not all of us have the luxury of teams of devs to help us sort out the new 'paradigm.' I just wasted about 2 days trying to figure out how to work with the new iteration of SDE, only to find that I no longer need it from a forum posting, rather than ESRI documentation. This is for a directed study with a student interested in learning to manage these types of architectures. He is quickly learning that, perhaps, the greatest takeaway will be forum mining skills.
In F, L & T,
Mr. Stacey D. Maples
GIS Specialist & Instruction Coordinator
Twitter = MapNinja
AIM = yalemapninja
GTalk = yale.map.ninja
Yahoo IM = stacemaples
ICQ = 57891396
Yale University Map Department
Sterling Memorial Library 7th Floor
130 Wall Street /P.O. Box 208240
New Haven, CT 06520-8240
The Yale University GIS Support Page [http://guides.library.yale.edu/GIS]
The Yale University Map Department [http://www.library.yale.edu/maps]
Yale GIS-L Mailing List Subscription [http://mailman.yale.edu/mailman/listinfo/gis-l]
Search for scanned maps and other Digital Objects from Yale's Library [http://discover.odai.yale.edu/ydc/]
Yale GIS Workshops Calendar [http://bit.ly/xqPvdS]
"I have a map of the United States... actual size.
It says, "Scale: 1 mile = 1 mile."
I spent last summer folding it."
I have to admit, as someone who has been managing SDE installs since 9.1, I didn't pick up on this paradigm shift until a few days ago when I did a test 10.1 on Oracle install, and realized that with Toad and the Python interpreter in ArcCatalog I could pretty much do everything I needed to get the enterprise geodatabase created, a user-schema geodatabase created, and the .sde connection files to access them created. I think that is a good step forward for basic installs. If I can take the SDE Application server out of the picture, that is a big time saver. The toolbox tools are a little rough, but with arcpy you can get the job done.
It would be great to have access to some more of the sde app server command line tools available via the python arcpy module. I'm thinking especially of the tools like sdeconfig and sdedbtune, which still seem necessary to me. I hope that is in the works. If I'm missing something on that front, I would love to be educated.
Looking at it now, the progression makes sense and I'm glad for the new options.
I haven't had much experience with ArcSDE or ArcGIS server in general, and I have one question. While I was reading through the tutorial on ArcGIS API for Silverlight (I believe that was before 10.1 was released) it was mentioned somewhere that to be able to edit a feature class using a web application, the feature class must be stored in an SDE database and published as a feature service. Is the part regarding SDE still true at 10.1?
Our office is finally getting an ArcGIS sever (Standard Edition at the Workgroup level) so we can play with it and look into web map application development. The system admin is going to set it up soon, and the ArcGIS Server 10.1 SP1 installer is all we've got (no Web Adaptor or ArcSDE installer/setup). Is the following scenario for testing purposes possible? If not, what else do we need to install/configure, or what is the alternative?
Install ArcGIS for Sever on sever A;
Using ArcGIS for Desktop on a workstation, create a file geodatabase on either sever A or some other server B on the same office LAN, and create a feature class in the FGDB and publish it as a feature service;
Create a web mapping application and host it on server A (separate IIS set up necessary?);
Display, query, and edit the feature class through the web application over the internet.
If this scenario is possible, does ArcGIS Server create a copy of the feature class and serve the copy when I publish the FC as a service? If that is true, will I be able to access/edit this copy in ArcGIS for Desktop on the workstation?
Last edited by xyknewtry; 03-26-2013 at 02:10 PM.
You will need the feature class stored in SDE in order to create a feature service. The ArcGIS for Server Workgroup (Standard Edition) comes with SQL Server Express. You should be able to download the installation files from the customer care portal:
After you install SQL Server Express, you will just need to enable SQL Server Express to store geodatabases:
Additional helpful links:
Setting up a connection:
Adding a database server to Desktop:
Creating a geodatabase on a database server:
ArcGIS for Server is, just like ArcGIS for Desktop, fully capable of using both an ArcSDE Application Server, or Direct Connect, for connecting to an enterprise geodatabase.
From a functionality point of view, there is absolutely no difference between using an Application Server or Direct Connect. So yes, "Direct Connect" is sufficient for ArcGIS for Server to create (editable) Feature Services.
"The ESRI Geodatabase Framework" PDF
Please post any comments to the document in that thread and not here.