Document versioning
Store many versions of the same document, download or revert to a previous version.
Released: June 29, 2018
Historically, Mayan EDMS has steered away from adding too much JavaScript in its code. The goal was to be able to maintain a robust, backend-based page rendering method that will be as future-proof as possible. This approach comes at the cost of some page loading speed, and reduced user interface interactivity.
The whole system has been update to work as a modern Single Page App. Single Page Applications (SPAs) rewrite the current page dynamically rather than loading the entire page on each click of the mouse. This makes the web application feel and behave more like a desktop application. Because the majority of the styling and JavaScript code is loaded only once, there is also the added benefit of less data down the wire. Thus the application becomes lighter and provides a faster response time to user events. Because the style is loaded and interpreted at the beginning, the browser is also able to apply it to the new content faster.
This redesign was achieved using only HTML5 and jQuery. Aside from two additional jQuery libraries, there are no extra framework dependencies. With the conversion to an SPA, many other petitions for user interface improvements are now possible.
The move to Django 1.11 proved to be a real challenge. Even though Django 1.11 is a minor release, it breaks compatibility and interfaces in several key areas. Among these were templates and form widgets.
Mayan EDMS uses a complex template, form and widget system. The system mimics object-oriented concepts like inheritance at the rendering stage. This allows the more than 300 views to be serviced with just a handful of forms classes and base templates. Testing and auditing all the views and forms after the upgrade was a lot of work.
Along with the upgrade to Django 1.11, many deprecations warning were fixed in preparation for an eventual upgrade to Django 2.0.
These work by allowing users to subscribe to a particular event like Document Uploads or to an event of a particular document like when an invoice is edited. If these events occurs, the user gets a reminder next to the bell icon in the main menu bar.
Most of the requirements, dependencies and libraries were upgraded to their latest release.
Searching without using a specialized search database is difficult. Mayan’s design calls avoiding a separate search engine at the cost of some missing search syntax. The OR and the negative term support are the first attempts at adding special syntax to Mayan’s search code.
By default now, search terms are routed to an “AND” query. That means that a search for:
Tag1 Tag2
will only return documents with both tags attached. To offer the opposite choice we added an “OR” syntax. Searching for:
Tag1 OR Tag2
will return documents with either tag attached.
Support for literals terms was also added.
Searching for:
blue car
will return documents with the words “blue” and “car”, even if they are not together. That means getting documents with the phrases “blue sky” and “slow car”. To search for exact terms enclose them in quotes:
“blue car”
This will return only documents with the exact phrase “blue car”.
If you've ever tried running two instances of Mayan EDMS, you would
have noticed that they both try to create a lock file in the /tmp
directory with the same name. Only the first instance will be able to
run.
The lock filename needs to be unique to each instance, yet predictable so that the workers of an instance can also access the same lock file.
This issues was solved by using a hexadecimal hash representation of the
installation’s unique SECRET_KEY setting. The use of a hash makes
reversing the value in order to obtain the SECRET_KEY impossible for
all practical purposes.
Display sizes for document display, previews, and thumbnail were specified as a string that included the horizontal and the vertical resolution separated by the character “x”. Using an “x” character to separate resolution elements is not standard.
This version splits the settings for specifying resolutions into two settings for each size. One setting for horizontal resolution and another for vertical resolution.
The settings are now:
DOCUMENTS_DISPLAY_WIDTH, DOCUMENTS_DISPLAY_HEIGHT,
DOCUMENTS_PREVIEW_WIDTH, DOCUMENTS_PREVIEW_HEIGHT,
DOCUMENTS_PRINT_WIDTH, DOCUMENTS_PRINT_HEIGHT,
DOCUMENTS_THUMBNAIL_WIDTH, DOCUMENTS_THUMBNAIL_HEIGHT
The steps needed to upgrade a document using form-tools' SessionWizard were hardcoded in the source app. This made it very difficult to add or remove wizard steps.
The steps of the wizard are now defined by a new class called
sources.wizard.WizardStep. The existing steps to select a document
type, enter metadata and tag the document, have been converted to
function as WizardSteps subclasses. The converted steps now live in
sources.wizards.WizardStepDocumentType,
tag.wizard_steps.WizardStepTags, and
metadata.wizard_steps.WizardStepMetadata.
The steps need to define the following methods:
done: This method is execute when the wizard finished the last
step an enter the step where the actual file are uploaded. This
steps is used to encode form data into the URL query string that
will be passed to the document upload view for each file uploaded.condition: This method is used to display the step conditionally.
If this method return True it will be displayed during the upload
wizard execution. To skip the step, return False or None.get_form_initial: This method is used to return the initial data
for the step form. Use this method to set up initial values for the
step's form fields.step_post_upload_process: This method will be executed once the
document finishes uploading. Use this method to process the
information encoded in the URL querystring by the step's done`
method.Once the WizardStep subclass is defined, it needs to be registered.
This is done by calling the .register method of the WizardStep class
with the subclass as the argument. Example:
WizardStep.register(WizardStepMetadata)
This statement must be located after the subclass definition. Finally,
the module defining the wizard step must be imported so that it is
loaded with the rest of the code and enabled. The best place to do this
is in the .ready method of the apps' apps.py module. Example:
class TagsApp(MayanAppConfig): has_rest_api = True has_tests = True name = 'tags' verbose_name = _('Tags') def ready(self): super(TagsApp, self).ready() from actstream import registry from .wizard_steps import WizardStepTags # NOQA
The WizardStep class also allows for unregistering existing steps. This
is accomplished by calling the .deregister method of the WizardStep
class and passing the subclass as the argument. This method should also
be called inside the .ready method of an apps' apps.py module.
Example:
class TagsApp(MayanAppConfig): has_rest_api = True has_tests = True name = 'tags' verbose_name = _('Tags') def ready(self): super(TagsApp, self).ready() from actstream import registry from metadata.wizard_steps import WizardStepMetadata # NOQA from sources.wizards import WizardStep # NOQA from .wizard_steps import WizardStepTags # NOQA WizardStep.deregister(WizardStepTags)
This will cause the tags assignment step to not be assigned to the upload wizard anymore.
Using the new WizardStep class a new upload wizard step was added to
assign documents being uploaded to any number of cabinets while being
uploaded. This step was been assigned number 4 in the order of step for
uploading a file.
On some devices the height of the pages on the document preview view were squashed. The CSS for this view was updated to fix this issue.
After reports that it is not working in 100% of the cases, the feature that detects and fixes the orientation of PDF has been marked experimental and now defaults to being disabled.
The Mayan EDMS Request for Comments or MERCs will be use to propose and or document the new features, the existing code, and the processes governing the project. MERCs 1 and 2 have been approved. MERC-1 outlines the MERC process itself and MERC-2 documents the way API tests are to be written for Mayan EDMS.
The duplicated documents system has been improved to also better detect when the duplicate of a primary document has been move to the trash. In this instance the duplicate count of the primary document would be zero and will cause the primary document to not show in the duplicated document list view.
If the duplicated document is deleted from the trash the system now will launch a background clean up task to permanently delete the empty primary document's duplicate document entry from the database.
It is now possible to pass arguments to the document, document cache and
document signatures storage backends. To pass the arguments, use the new
settings: DOCUMENTS_STORAGE_BACKEND_ARGUMENTS,
DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS, and
SIGNATURES_STORAGE_BACKEND_ARGUMENTS.
The FileBasedStorage driver originally provided has been removed. With
this change the setting STORAGE_FILESTORAGE_LOCATION has also been
removed. The storage driver now default to Django's own
FileSystemStorage driver. By using this driver each app is responsible
of specifying their storage path. The path path (or location) is
configure via the DOCUMENTS_STORAGE_BACKEND_ARGUMENTS,
DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS, or
SIGNATURES_STORAGE_BACKEND_ARGUMENTS for the documents, document cache
and document signatures respectively.
For example, to change the document storage location use:
DOCUMENTS_STORAGE_BACKEND_ARGUMENTS = '{ location: <specific_path> }'
If no path is specified the backend will default to
mayan/media/document_storage.
Finally, to standardize the way app use storage, the storages.py
modules is now used instead of the runtime.py module.
When viewing the event list, the Actor (user) column is not displayed as a link. Clicking this link will filter the event list and display the events performed by that user. The view of event for each user can also be viewed using a new link added to the user list view in the setup menu.
A faster way to select multiple item has been added. Click the checkbox of the first, hold the Shift key, and then click the checkbox of the last item of the selection. This will select the first, the last and all items in between. To deselect multiple items the same procedure is used. This code was donated by the Paperattor project (www.paperattor.com).
An internal utility to install and upgrade the JavaScript dependencies was added. This depency manager allows for the easier maintenance of the JavaScript libraries used through the project.
Previously JavaScript libraries we downloaded and installed by manually. These libraries were them checked into the Git repository. Finally to enable them the corresponding imports were added to the base templates in the appeance app.
This new manager is the first step to start resolving these issues. The manager allows apps to specify their own dependencies. These dependencies are then downloaded when the project is installed or upgraded. As such they are not part of the repository and lower the file size of the project.
Removing a document type from a workflow will now also remove all running instances of that workflow for documents of the document type just removed.
To facilitate the inclusion of submissions provided by third parties, the project has adopted the use of individual and entity contributor assignment agreements. These agreements make clear the process to transfer the rights to submissions. With these agreements in place we now have a documented and legally sound method to accept submissions that we couldn't before.
Starting with version 3.0, a warning message will be shown in the console and in the user interface when using SQLite as the database engine. When it comes to Mayan EDMS, SQLite should only be used for development or testing, never for production. This is due to Mayan EDMS exceeding the concurrency capabilities of SQLite. The results are duplicated documents, frequency database locked errors, among other issues. Suggested database backends are PostgreSQL and MySQL (or MariaDB) using a transaction aware storage engine like InnoDB.
Parsing email messages is a complex task. To increase compatibility with the many interpretations of the standards that govern email messaging, Mayan EDMS now uses Mailgun's flanker library (https://github.com/mailgun/flanker). Thanks to flanker, Mayan EDMS now gains new capabilities when it comes to parsing incoming email. For example, in addition to mail attachments, it is now possible to process files included in emails as inline content.
Document view permissions is nor required to view the details
of a trashed document.Workflow view permission for the workflow to be
able to view a document type's workflow list.mayan_object_permissions
instead of the mayan_view_permissions dictionary.Workflow view permission to view the workflow
instance list.Workflow view permission to view a workflow
instance's details.ALLOWED_HOSTS settings now defaults to a safer
['127.0.0.1', 'localhost', '[::1]']MAYAN_SECRET_KEY,
MAYAN_CELERY_ALWAYS_EAGER, MAYAN_CELERY_RESULT_BACKEND,
MAYAN_BROKER_URL, MAYAN_DATABASE_ENGINE,
MAYAN_DATABASE_CONN_MAX_AGE, MAYAN_DATABASE_NAME,
MAYAN_DATABASE_USER, MAYAN_DATABASE_PASSWORD,
MAYAN_DATABASE_HOST, MAYAN_DATABASE_PORT, MAYAN_DEBUG.ALLOWED_HOSTS
to ['127.0.0.1', 'localhost', '[::1]'].SECRET_KEY and VERSION files.COMMON_PRODUCTION_ERROR_LOGGING setting to control the logging
of errors in production. Defaults to False.DOCUMENTS_LANGUAGE_CHOICES setting option. Replaced
with the new DOCUMENTS_LANGUAGE_CODES.DOCUMENTS_LANGUAGE_CHOICES setting option. Replaced with
DOCUMENTS_LANGUAGE_CODES.The newly added 'flanker' dependency used to process email, produces a number of warnings on the console that are imposible to turn off. These are not critical and are related to coding practices in the library. All warning from flanker can be ignored.
Example: "WARNING:flanker.addresslib._parser.parser:Symbol 'domain' is unreachable"
Type in the console:
$ pip install mayan-edms==3.0
the requirements will also be updated automatically.
If you installed Mayan EDMS by cloning the Git repository issue the commands:
$ git reset --hard HEAD $ git pull
otherwise download the compressed archived and uncompress it overriding the existing installation.
Next upgrade/add the new requirements:
$ pip install --upgrade -r requirements.txt
Migrate existing database schema with:
$ mayan-edms.py performupgrade
Add new static media:
$ mayan-edms.py collectstatic --noinput
The upgrade procedure is now complete.
<!-- -->
[:github-issue:`264`]{#id2 .problematic} migrate fails on document_states 0004_workflow_internal_name
::: {#id1 .system-message}
System Message: ERROR/3 (<string>{.docutils}, line 667);
backlink
Unknown interpreted text role "github-issue". :::
[:github-issue:`269`]{#id4 .problematic} Lack of authentication for document previews
::: {#id3 .system-message}
System Message: ERROR/3 (<string>{.docutils}, line 668);
backlink
Unknown interpreted text role "github-issue". :::
