Model Builder - Not Nullable Error Suddenly Occurring

[SStrand]

I have a model builder process that has worked flawlessly for about two years now. It takes a .DBF file and modifies field names and redistributes their order. The process is run once a month. Today I ran the newest file and every occasion where a field was added in model builder and the DBF has empty values, it is giving me an error for "Field is Not Nullable."

I understand this is an easy fix by going into each "add field" process and allowing null values, but the entire model is 120+ processes, with at least 1/4 of them being "add field." Also my work computer is extremely slow and once I click "OK" to change a setting in model builder, it has to think for about 20 seconds before allowing me to do anything else.

I have compared this months dbf to others and there are no differences within the table, so previous versions have had no issue with empty values. The only thing I can think of that has changed in the past month is that I have upgraded to the latest service pack.

I just wanted to know if anyone has an idea of what could have happened, and maybe if there is a quick fix to the null value issue. Thanks!

Steve

[curtvprice]

This error message occurs when you try to add a nullable field to a non-geodatabase feature class. I'm wondering if your model is working on a temporary dataset in a folder scratch workspace when previously this was a geodatabase scratch workspace.

Something else I thought of is perhaps the field was there before (which makes the add field skip processing with a warning) and now it isn't there so it's now actually trying to create the field, causing the error mesage.

Since the model builder is 120+ processes to do this restructuring of the table, maybe this is a good opportunity to re-engineer your process using the Merge tool with a field map (created with Calculate Value) may be a more efficient and maintainable approach then adding, dropping, and calculating fields.

[SStrand]

Hi Curt,

Thanks for the tips. Where I work the standard is still shapefiles and DBF tables so no part of the model touches anything geodatabase related. In regards to the second idea, I could see that being an issue but it is occurring with newly added fields as well, which were in no way a part of it before being added.

In regards to the last suggestion, when I original made the model I tried to use the merge tool to field map and it didn't work. I found countless forum posts about how that was a broken tool in that sense and so I gave up on it. Does it now work correctly? I would LOVE to rebuild the model and cut it down to just a few steps! I'll have to spend an afternoon trying it!

This error message occurs when you try to add a nullable field to a non-geodatabase feature class. I'm wondering if your model is working on a temporary dataset in a folder scratch workspace when previously this was a geodatabase scratch workspace.

Something else I thought of is perhaps the field was there before (which makes the add field skip processing with a warning) and now it isn't there so it's now actually trying to create the field, causing the error mesage.

Since the model builder is 120+ processes to do this restructuring of the table, maybe this is a good opportunity to re-engineer your process using the Merge tool with a field map (created with Calculate Value) may be a more efficient and maintainable approach then adding, dropping, and calculating fields.

[curtvprice]

Thanks for the tips. Where I work the standard is still shapefiles and DBF tables so no part of the model touches anything geodatabase related.

If you do not specify a scratch workspace, in Arc 10 the default scratch workspace (usually) is the current map document's default geodatabase, which by default is a geodatabase in your My Documents folder. I highly recommend you make sure the map document is set up with a defined workspace and scratch workspace -- and since you are only working with shapefiles, this should be a folder.

In regards to the last suggestion, when I original made the model I tried to use the merge tool to field map and it didn't work. I found countless forum posts about how that was a broken tool in that sense and so I gave up on it. Does it now work correctly? I would LOVE to rebuild the model and cut it down to just a few steps! I'll have to spend an afternoon trying it!

Field maps are tricky anyway, and difficult to manipulate in model builder -- though you can if you are willing to develop Python code within the Calculate Value tool. It would be nice if there were some geoprocessing tools that provide a better interface for working with field maps in Model Builder (and Python!).

A poor-man's method of field mapping I have used is to create an empty feature class (shapefile) with just the fields I want and then use the Append tool with NOTEST - fields not included in the output will simply be discarded. A way to do this efficiently is to use the Create Feature Class tool to create an in_memory temp feature class ("in_memory/tmpPoly") and then run AddField/DeleteField in Python code (Calculate Value) to set up the schema (field layout) you need - this works fast on an in memory feature class. Then Copy Features this empty feature class to a shapefile and Append (NOTEST) your data to it. As I said, writing Python code to manipulate field maps in Calculate Value is the "real" way to do it, but it is not trivial coding IMHO.



I know it seems like a pain to write Python code in ModelBuilder, but when you get more than fifty tools exposed in a model, it can take a very long time to validate the model (as you have found). It would be much more efficient to put your many AddField commands inside a Calculate Value tool as I have in the above example.

If you agree that validating models is way too slow, please vote up this idea on ideas.esri.com (you need to be logged in for the link to work):

Speed up Toolbox load in ArcGIS 10
https://c.na9.visual.force.com/apex/...0000000c1uFAAQ