Released: June 13, 2019
Work on version 4.0 continues along. Version 4.0 brings so many changes that it missed its release schedule. Therefore we decided to release an interim version, this one. This version includes bug fixes and back-ports many improvements already in the 4.0 development version. It also helps bridge the gap between the internal code and interfaces between the 3.x and 4.x code branches. The minor increment means that user-facing backward incompatible changes are minimal. Most of the changes are internal and of notice for developers.
That doesn't mean there are no changes important to users. This release brings about one third (1/3) of the user interface changes that have already landed in the 4.0 version.
Besides the usual template and style tweaks, this version adds a new contextual navigation bar. This bar is a hybrid between a list menu bar and a sidebar menu. Icons in this menu object will show in the object list view and when the object is the main object in the template. This reduces substantial mouse travel and clicks, as most views of the object now remain "open" in the sidebar instead of being hidden inside the "Actions" drop down.
The "Actions" drop-down will now divide the available actions links depending on the menu that defines them ("action", "secondary") and the object that they act upon. This is useful on views that can display more than one object at a time such as the setup views. During the workflow setup view, now the action links will be split between action for the workflow, for the state, and for the transition making setup navigation much faster and requiring less mental effort as the user no longer need to remember which link affects which view object.
Another big user interface change that landed on this version is the ability to sort lists of objects by column fields. Sortable fields will have their column heading displayed as an HTML link. Clicking on a column heading link will sort the list by the values of the column. Clicking the heading again will invert the sort order. A small arrow icon will show the sort order.
A new app was added to the core. The app is named "File metadata" and allows extracting file information. The app includes a driver by default to extract EXIF field information. This file field includes camera information for photos, authors for office documents and PDFs and other information that can now be used to search or index documents. This app was previously a separate app called Mayan-EXIF (https://gitlab.com/mayan-edms/exif). The app was generalized and is now part of the core group of apps.
This is a "minor" release only in version number scheme since it includes a big number of code changes. The minor number increase means that it will be as backward compatible with version 3.1.10 as possible.
The API documentation generation code was moved from the root module where it was out of place to the REST API app. The published URLs however remain the same and backwards compatible.
The dependencies app also has the ability to introspect packages and extract copyright and license information. This removes the need to include such texts explicitly in the code for all packages.
Dashboard app: All the code to defined and generate the dashboard was moved from the common app into its own app called "dashboards". The functionality remains the same but along with the formalized interface, the move will allow future planned improvements to the dashboard functionality to be rolled out. While only one dashboard will still be displayed, it is possible to define in code more than one dashboard using the Dashboard class.
Autoadmin app: The autoadmin app which is in charge of creating the initial admin user after the installation, has been included in the core. This app is made by the same author of Mayan EDMS and at one time in the past was part of the core apps.
File metadata: Allows extracting file information. The app includes a driver by default to extract EXIF field information (https://en.wikipedia.org/wiki/Exif). This is mostly multimedia information but the default driver can also extract information from authorship from PDF and office files. The information extracted by this app can be used to search or index documents.
Internally many interfaces and calls have been deprecated in other to move the code closer to 4.0's standard. The only feature deprecated on the user facing side is the convertdb management command. This command was added to allow existing installation using SQLite as the database manager to convert their database to one of the recommended database managers.
After many reports the consensus was reached that this functionality is not meant to be provided in the project. Software projects have little or no control of the aspects upon which they rely. Framework, environment, platform, OS, databases are such examples.
Database conversion is a task best suited for operations oriented software and professional and no Mayan's end users. For these reason the database conversion command has been deprecated and will be removed in the next major version.
Docker image updated to use Debian 9.8.
The size of the Docker image was reduced from 1.25 GB to 1.09 GB.
Support was added for setting the Docker container user's UID and GUID.
The events system received many updates. Now more apps register and record events. Some new events recorded are:
A new link was added under the User menu to show all the events of the currently logged user.
Existing config.yml files need to be updated manually. The prefix 'mayan.apps' must be added to any reference of an app.
All the Internet Explorer specific HTML markup was removed from the templates.
Installation seems to be the most error-prone stage of the process in getting to use Mayan EDMS. Just like configuration, installation, and dependency management are other areas where the underlying framework and language don't provide tools and it is up to the project to come up with an alternative to bridge the functionality gap.
Besides the dependency check view in the Tools menu, a management
command was added to show the same information from the command line.
This command is called
Developers can also use the new
generaterequirements command to
generate the Python requirement files from the dependencies declared in
the apps of the project, keeping all dependency files synchronized with
Big or complex app modules were split into separate modules. This includes models, views, and tests. A star (*) was added to keep current imports working but it is encourage to import from the module itself.
For example: The document apps models.py module was split into:
The module documents/models/__init__.py include them all using a start import so that any import to the path mayan.apps.documents.models still works.
Other modules that were split were:
Development URL definitions were removed from the main URL definition
file. Instead, the top level
urls.py file was converted into a module
folder holding several URL pattern definition files. Each URL pattern
definition file can be specified independently by changing the value of
Instead of inserting the path of the apps into the Python path, the apps
are now referenced by their full import path. For example: Instead of
referencing the Documents app using the string
documents, the full
dotted path must be used,
mayan.apps.documents. This change namespaces
all apps under a module folder and solves name clashes with external or
native Python libraries. Once example of such name clash: The statistics
app was called "statistics" and it clashed with Python's new
statistics library of the same name. The solution at the time was to
rename Mayan's to "mayan_statistics". With this change solutions
like this won't be necessary.
This means that any reference to an app, either in the code or from a
configuration file must now be prefixed with
config.yml files need to be updated manually.
For developers, a new test mixin was added called
SilenceLoggerTestCaseMixin. It allows lowering the level of loggers
during tests. To use this mixin call
provide the dotted path to the module whose message level will be
lowered. By default the new level will be set to critical.
Support was added for link icon path imports. Instead of importing all
icons, a link can reference the dotted path of the icon. The link
argument for this is called
Support for link icon strings was removed. Only icon classes are allowed
now. This keeps all icon definitions encapsulated in the
Middleware were updated to support both, Django's old and new style middleware.
A new class named
FormOptions was added to reduce the boilerplate code
needed to add Meta options to custom Form classes. This class is used to
create a new reusable form that will filter a selection field based on a
permission. This reusable form is called
class is used to simplify and unify the
Support was added for help text on extra fields of the
Language choices generation was moved to
TwoStateWidget class was converted to work as a template widget.
It is also now compatible with
SourceColumn and don't need to be
evaluated using lambdas.
SourceColumn class received many updates to make is more versatile
and reduce the boilerplate code to define model columns. It now supports
related attributes using a double underscore separator. This eliminates
the need of functions or lambdas for related fields. Also added was
support displaying a placeholder text for empty attribute values. This
eliminates the need to create a model or class method to capture empty
or None values and display a translatable text. Support was added to
enable an instance's absolute URL for a column removing the need to add
and define a single use link for a clickable column. The
class now supports keyword arguments via
kwargs to pass to the column
The star import was removed from the ACL and Common app's
common.mixins imports have to done explicitly.
The authentication function views were updated to use Django's new class based authentication views.
The current user views were moved from the common app to the user management app. And vice versa the user and current user password change views were moved to the authentication app from the user management app. Now everything related to accounts resides in the user management app and anything relating to passwords and authentication resides exclusively in the authentication app.
The custom email form widget provided by
common.widget was removed as
Django now includes one.
All file related utilities were moved from the
to the to the storage app's
The navigation and authentication templates were moved to their respective apps. They are no longer found in the appearance app's templates folder.
General queryset slicing of the Document list view was removed and added to the only subclass view that uses it which is the Recently Added Document view.
A new view called
AddRemove was added which replaces
All views were updated and
AssignRemove removed from the code. This
class offers a much cleaner design and more functionality like adding or
removing multiple items from a single post request. This is achieve by
passing a queryset of the selection instead of calling the add or remove
methods for each item of the selection.
A new test case mixin was added to provide ephemeral test models. These are memory only model classes that allow tests to be performed much faster while testing all aspects as if it were any other statically defined model. Several test views that used documents models were converted for speed increases of several order of magnitude. Along with ephemeral models, ephemeral test permissions were added removing the need to use static permission for generic permission compliance tests.
Move stub filtering to the Document model manager. All subclasses of the
DocumentListView class were updated to simplify their document
The transformation manager methods were renamed from
get_for_object, to reflect the
fact that they can only operate only models.
The title calculation was converted from a template markup to a template tag.
The permission inherited computation was improved and now operates mostly at the database layer instead of being mostly a Python computation. A similar improvement was done to the ACL calculation system. The ACL calculation now operates mostly on the database layer freeing many resources and scaling better. The access checking method was updated to be a wrapper for the new ACL queryset calculation, reducing the amount of code and benefiting from the ACL queryset calculation improvements.
Related to the ACL calculation updates is the fact that views will now
operate of a filtered queryset as returned by
instead of directly checking the access of the action being performed by
the view. The update process of the views to this new methodology was
started and will be completed in the next versions. Eventually the
.check_access method will be either removed of used for special cases.
A small change with big repercussions is the rename of
get_source_queryset. This change represents a shift in the way
views will filter their queryset in the entire project. Since Django
doesn't provide hooks for transforming a queryset before usage by the
view, instead of being passed a model or a queryset directly, views will
be passed a source queryset via
get_source_queryset which will be
restrict_queryset and this final queryset will be passed
to the view as if from its normal
get_queryset method. This allows
using views as they are normally provided by Django and add Mayan's
access control with minimal changes.
Uniqueness validation was added
MultipleInstanceActionMixin was removed,
be used instead. These two classes provide action handling for single
pk) or multiple model instances ( using
id_list) from the
single view definition.
MultipleObjectMixin improvements were back ported. Selections are
ObjectListPermissionFilterMixin was removed.
license.py module in apps was removed, licenses are now defined in
dependencies.py for the ones that the dependency app can't introspect
Model permission proxy models support were removed. The permission
system can now detect inherited permission from multiple levels. Related
to this change the
related access control argument to
links, API views
mayan_permission_attribute_check was removed. This is
all now handled by the related field registration.
The permissions system now supports nested access control checking. Self
referencing models like
Cabinets, must provide a custom function to
filter their permission checking.
An optimization was added which removed the exception catch for the
permissions argument of the
argument must now be an iterable.
Signal handlers must now be prefixed with
.filter_by_access. Replaced by
Rename the form template
differentiate it from Python form classes.
All views now redirect to the
common app's home view instead of the
The project was updated to PyYAML version 5.1. The use of
safe_dump was changed to
dump using the
SafeLoader as fallback. This makes YAML handling faster and safer.
Mailing profiles were updated to allow specifying the email sender address. This change closes GitLab issue #522.
If an existing mailer profile specified a backend that is now invalid
instead of showing an error it will be replaced by a new
Due to the change of using the entire dotted path of apps, the backend of existing mailing profiles will be invalid and must be re-created.
The document link URL when mailed is now composed of the
COMMON_PROJECT_URL setting plus the document's URL instead of Django
Site domain. This was the only use of Django's
Site model and was
removed to an easier to configure setup.
Two new Mayan EDMS Requests for Comments were approved during version 4.0 development and applied to this release too.
MERC 5 now requires all callables to use explicit keyword arguments. This MERC is effect makes positional arguments obsolete. These are only retained for Python modules and callables that don't support named or keyword arguments.
If you are developing a third party app, update your non-access view tests to expect a 404 and not a 403 error.
The code audit performed during the development of version 4.0 revealed many areas where optimizations were possible. All the backward compatible optimizations were backported to this version. These are:
copyfileobjin several places.
As a result, the memory footprint and CPU usage were lowered substantially. Memory usage was lowered to 700MB of RAM under full load. This is great news for all user but of special importance for restricted environments like low tier virtual hosts, container deployments, and single board computers like the Odroid or the Raspberry Pi.
Add new and default Tesseract OCR backend was added to avoid Tesseract bug 1670 (https://github.com/tesseract-ocr/tesseract/issues/1670, https://gitlab.gnome.org/World/OpenPaperwork/pyocr/issues/104). This bug make it impossible to use Tesseract via PyOCR under Python 3.
The role permission grant and revoke permissions were removed. Instead only the role edit permission is required to grant or revoke permissions to a role.
Dual permissions are now required for some objects. For example, when adding or removing users to and from a group, the edit permission must be granted to the user and the group. These permission changes are minimal for this version but will be rolled out to be the norm on the entire system.
Long awaited Python 3 support is here. To ensure a smooth transition only the Python package will be release supporting Python 2.7 and 3. For the next release, the Docker image will be converted to work on Python 3. And finally, on the release of the next major version, version 4.0, Python 3 will be the only Python version support. This version of Mayan EDMS, as well as future versions of the same series (3.x) will be the last version supporting python 2.7.
If important security issues are found subsequent series 3.x releases will be done for users that can't for some reason upgrade to a version using only Python 3.
Database transaction handling was added in many more places to ensure data integrity even in extreme situations.
For each version, we identify functionality or requirements that can be removed. For this version, Django suit was removed from requirements as well as support for generating documents images in base 64 format.
HOME_VIEW setting was defined without a namespace and as a top
level setting. This configuration is reserved for native Django setting
HOME_VIEW setting is now namespaced to the COMMON app where it
is defined. The setting global name therefore changes from
More Django settings were exposed and can now be modified:
New default value of 65535 for the
This means that new documents will be read and process in blocks of 65K
to determine their SHA256 hash instead of being read entirely in memory.
Removal of the
MIMETYPE_FILE_READ_SIZE setting. A new method was
implemented to reduce memory usage of the
MIME type inspection code.
Instead of limiting the number of bytes read as specified by the
MIMETYPE_FILE_READ_SIZE setting, now the entire file is saved to a
temporary file and the
MIME type library called with the temporary
file reference. This approach while minimally slower provides the
benefits of lower memory usage without sacrificing
MIME type detection
accuracy which was a downside of the
Several improvements were back ported to the search app. One of this
allows returning the search result as a queryset. Queryset are "lazy"
and not evaluated until accessed. This means a queryset can represent a
vast number of documents with consuming the entire memory that would be
required to hold all the documents instances as a list would. This
change make the memory limiting setting
SEARCH_LIMIT obsolete and was
Additionally the search time elapsed calculation was removed. This code stopped being used from the code several version ago.
The default value for the recently added, recently accessed, and favorite documents settings was increased from 40 to 400. Using the default pagination size of 40 documents per page than means a total of 10 pages of documents for each of one of these views instead of just one page.
COMMON_TEMPORARY_DIRECTORY was moved to the storage app.
The setting is now called
Many user interface updates were back ported some very noticeable others less so but still have a big impact on usability and navigation.
New icons classes and templates were added which allow many icon manipulations like icon composition. This allow for a more unified visual language.
The right side menu, called the "sidebar" is now a separate unit in both markup and code. This new side bar is implemented independently of the main view. This change allows for independent scrolling and more usable screen area.
Support was added for sortable columns. These allow sorting the results of a list by clicking on the column heading label. Clicking on the label again reverses the sorting order.
select2 widget is used in more places reducing data entry and providing quick search capabilities.
The markup for the invalid document image was converted into a server side template for easier customization and reuse.
sidebar menu instance was removed. The
secondary menu now
perform the same function.
For every function that involved two types of objects, two views were added. For example: Associating a workflow to a document type can now be done from the workflow edit view or from the document type edit view. This eliminates jump over from the view of one object type to the other.
Form can be submitted by pressing the Enter key or by double clicking.
The full name field was removed from the user list. Instead the first name and last name fields are shows which are sortable.
Completion level and initial workflow state are now shown in the workflow proxy instance menu.
Fix translation of the source upload forms using the drag and drop dropzone.js widget.
Bootstrap and Bootswatch were updated to version 3.4.1.
Support was added for attaching form button aside from the default submit and cancel. The new add and remove views use this allow adding all or removing all with a single click.
CSS updates were added to maximize usable width of the screen.
Navigation error messages display was improved for easier debugging.
The permission list was removed from the ACL list view. This reduces clutter and unpredictable column size. The permission list can be accesses by pressing the "Permissions" button.
The default title truncation length was increased to 120 characters. Even when truncated, hovering over the title will now show the entire title as a pop over.
Previously form widgets and HTML widgets resided in the same .widgets.py module. The .widgets.py module is now reserved for form widgets and HTML widgets will be found in a new module called html_widgets for each app.
The interface for Mayan's HTML widget has been formalized and these can
be used with the
SourceColumn class without having to use a lambda.
The widget argument was added to the
SourceColumn was for this
ChoiceFormto be full height.
.queues.pymodule is now loaded automatically.
Remove deprecated requirements:
$ curl https://gitlab.com/mayan-edms/mayan-edms/raw/master/removals.txt | pip uninstall -r /dev/stdin
Type in the console:
$ pip install mayan-edms==3.2
the requirements will also be updated automatically.
If you installed Mayan EDMS by cloning the Git repository issue the commands:
$ git reset --hard HEAD $ git pull
otherwise download the compressed archived and uncompress it overriding the existing installation.
Remove deprecated requirements:
$ pip uninstall -y -r removals.txt
Next upgrade/add the new requirements:
$ pip install --upgrade -r requirements.txt
Perform these steps after updating the code from either step above.
Make a backup of your supervisord file:
sudo cp /etc/supervisor/conf.d/mayan.conf /etc/supervisor/conf.d/mayan.conf.bck
Update the supervisord configuration file. Replace the environment variables values show here with your respective settings. This step will refresh the supervisord configuration file with the new queues and the latest recommended layout:
MAYAN_DATABASE_ENGINE=django.db.backends.postgresql MAYAN_DATABASE_NAME=mayan \ MAYAN_DATABASE_PASSWORD=mayanuserpass MAYAN_DATABASE_USER=mayan \ MAYAN_DATABASE_HOST=127.0.0.1 MAYAN_MEDIA_ROOT=/opt/mayan-edms/media \ /opt/mayan-edms/bin/mayan-edms.py platformtemplate supervisord > /etc/supervisor/conf.d/mayan.conf
Edit the supervisord configuration file and update any setting the template generator missed:
Migrate existing database schema with:
$ mayan-edms.py performupgrade
Add new static media:
$ mayan-edms.py preparestatic --noinput
The upgrade procedure is now complete.
Due to the change of using the entire dotted path of apps, the backend of existing mailing profiles will be invalid and must be re-created.
Paths to apps must be updated in existing
mayan.apps. to any app reference. Some instances:
Check the supervisord logs at
/var/log/supervisor for additional
errors in the form:
ImportError: No module named ocr.backends.pyocr
To collect and compress the static media files, use the new
preparestatic command instead of the traditional
Both work the same way, but
preparestatic has a default blacklist
to avoid collecting files from tests, development setup, and demos.