Data Import API

[JohnHays]

CONTENTS DELETED

The author has deleted this message.

[Stephen Rufle-2]

I think much of the functionality you would need to write custom import
code would be in
org.ofbiz.webtools.WebToolsServices

I have also recently have to deal with large xml import files. My trick
was to break up the large file by entity.

Orig file: 238 mb had many different entities
Created a Party only file: 9.8 mb

There was also talk in a different thread about how the xml import
utilities use a sax parser so large datasets may actually import without
issues.


John Hays wrote:

> I have 100s of thousands of data items to put into the ofbiz schema
> (MySQL to MySQL) -- I have been playing with the XML import under
> webtools by writing the customer -> party data out to XML.  With the
> size of the datasets, this isn't very practical.  Is there
> documentation on how to do this with an API?
>
>
> John D. Hays
>
>
>
>
> www.mavericklabel.com
> 120 West Dayton Street
> Edmonds, WA 98020-4180
>
>

[BJ Freeman]

you can also assign a datasource in the entityengine.xml make sure not
to have the table update on, then use the webtools to create entities
for you current DB.
https://demo.ofbiz.org/webtools/control/view/ModelInduceFromDb
or
http://docs.ofbiz.org/display/OFBIZ/FAQ+-+Tips+-+Tricks+-+Cookbook+-+HowTo#FAQ-Tips-Tricks-Cookbook-HowTo-ExternalDatabase
BTW the FAQ are a good place to find things like this
here are some other options.
http://docs.ofbiz.org/display/OFBIZ/Handling+of+External+data
John Hays sent the following on 5/1/2009 11:29 AM:

> I have 100s of thousands of data items to put into the ofbiz schema
> (MySQL to MySQL) -- I have been playing with the XML import under
> webtools by writing the customer -> party data out to XML.  With the
> size of the datasets, this isn't very practical.  Is there documentation
> on how to do this with an API?
>
>
> John D. Hays
>
>
>
>
> www.mavericklabel.com
> 120 West Dayton Street
> Edmonds, WA 98020-4180
>
>

--
BJ Freeman
http://www.businessesnetwork.com/automation
http://bjfreeman.elance.com
http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro
Systems Integrator.

[David E Jones-3]

The SAX parser to import without memory issues has been in place for a  
few years. Generally imports of hundreds of MBs will be fine (just  
watch out for fk issues... and there are various options for dealing  
with these in the import screens).

-David


On May 1, 2009, at 12:50 PM, Stephen Rufle wrote:

> I think much of the functionality you would need to write custom  
> import
> code would be in
> org.ofbiz.webtools.WebToolsServices
>
> I have also recently have to deal with large xml import files. My  
> trick
> was to break up the large file by entity.
>
> Orig file: 238 mb had many different entities
> Created a Party only file: 9.8 mb
>
> There was also talk in a different thread about how the xml import
> utilities use a sax parser so large datasets may actually import  
> without
> issues.
>
>
> John Hays wrote:
>> I have 100s of thousands of data items to put into the ofbiz schema
>> (MySQL to MySQL) -- I have been playing with the XML import under
>> webtools by writing the customer -> party data out to XML.  With the
>> size of the datasets, this isn't very practical.  Is there
>> documentation on how to do this with an API?
>>
>>
>> John D. Hays
>>
>>
>>
>>
>> www.mavericklabel.com
>> 120 West Dayton Street
>> Edmonds, WA 98020-4180
>>
>>
>