Schedule full import periodically

Hi,
in our EazyBI installation, we have some accounts with a very large amount of data. To avoid adding not relevant data, we added a JQLfilter query that gets the issue created >= startOfMonth(-12).

The problem is that periodically we have to manually empty the account and reimport from scratch. We did this just yesterday this account went from 144,234 MB to 10,284.2 MB, freeing up more than 100GB of space on the filesystem.

For obvious reason, it is not possible to perform this reimport during working hours. So, I was wondering if there’s a way to trigger or schedule this action automatically.

Thanks,
Claudio

Hi @klawde

Great, adding a JQL query to limit imported data is a way to go.
Though, usually, issues that do not meet the JQL conditions, are deleted during each data import, so you should have only those issues created within the given time period.
Would you mind checking imported issues: are there issues created more than 12 months ago after the regular data import?

Another thing is growing database size. In the case of PostgreSQL, there is a problem related to how PostgreSQL handles garbage collection ( see 23.1.2 ) - in short, after insert/update/delete, nothing is removed from the database until auto or manual VACUUM is executed. As eazyBI import is doing a lot of DB operations and the default autovacuum settings are not very optimized for that, then as a result, the DB size could grow.
If this could be your case (growing DB size, PostgreSQL DB, imported only needed issues), please contact support@eazybi.com for further assistance.

Best,
Ilze / support@eazybi.com