In Django you cannot add or remove through= on M2M fields

Using native migrations in Django has been a very positive experience. Now and then however, some thing appear difficult or impossible. In this post, I will demonstrate how migrations can be used to move from an automatic ManyToMany relation in Django to an explicit relation. Lets start by creating a small app with a ManyToMany relation.

	from django.db import models
	class Bar(models.Model):
	spam = models.EmailField()
	class Foo(models.Model):
	bars = models.ManyToManyField(Bar)

view raw models.py hosted with ❤ by GitHub

Suppose we want some way to order the list of Bar instances in the relation. In setting up the ManyToMany relation, Django has created a table for us, named through_foo_bars. We can see what it looks like in python using the inspectdb command

$ ./manage.py inspectdb through_foo_bars

We can make explicit the automatic relation that Django has created in the form of an extra table, by adding the model to our application.

	from django.db import models
	class Bar(models.Model):
	spam = models.EmailField()
	class Foo(models.Model):
	bars = models.ManyToManyField(Bar, through='ThroughFooBars')
	class ThroughFooBars(models.Model):
	foo = models.ForeignKey(Foo, models.DO_NOTHING)
	bar = models.ForeignKey(Bar, models.DO_NOTHING)
	order = models.PositiveIntegerField(default=0)
	class Meta:
	db_table = 'through_foo_bars'
	unique_together = (('foo', 'bar'),)
	ordering = ('order',)

view raw models.py hosted with ❤ by GitHub

Note the ordering we specified in the Meta definition. We told Django to use our new model for the relation by specifying it as the through argument in the ManyToManyField. Django is happy to create a migration for us, incorporating the change.

$ ./manage.py makemigrations

The problem starts when we try to execute the migrations

$ ./manage.py migrate
...
ValueError: Cannot alter field through.Foo.bars into through.Foo.bars - they are not compatible types (you cannot alter to or from M2M fields, or add or remove through= on M2M fields)

So according to Django this kind of migration is not possible. Let's analyse what the problem is. The below migration tries to create a new model called ThroughFooBars and then change the ManyToManyField on Foo.

	# Generated by Django 2.2 on 2019-04-22 11:25
	from django.db import migrations, models
	class Migration(migrations.Migration):
	dependencies = [
	('through', '0001_initial'),
	]
	operations = [
	migrations.CreateModel(
	name='ThroughFooBars',
	fields=[
	('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
	('foo', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='through.Foo')),
	('bar', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='through.Bar')),
	('order', models.PositiveIntegerField(default=0)),
	],
	options={
	'db_table': 'through_foo_bars',
	'unique_together': (('foo', 'bar'),),
	'ordering': ('order',),
	},
	),
	migrations.AlterField(
	model_name='foo',
	name='bars',
	field=models.ManyToManyField(through='through.ThroughFooBars', to='through.Bar'),
	),
	]

view raw 0002_through.py hosted with ❤ by GitHub

The problem is ofcourse that no new model needs to be created because the table allready exists. In this case we DO want to change the migration state, altering the relation, but we do NOT want to change the database schema. We are lucky that django offers such migrations by providing us with migrations.SeparateDatabaseAndState. We will first modify the state to make the relation explicit, after which we can do the modifications to the schema to add the ordering. In the migration below, you can see we first modify the state, making the relation explicit in the state only, without modifying the database schema. Next we add the order field and the ordering Meta definition.

	# Generated by Django 2.2 on 2019-04-22 12:38
	from django.db import migrations, models
	class Migration(migrations.Migration):
	dependencies = [
	('through', '0001_initial'),
	]
	state_operations = [
	migrations.CreateModel(
	name='ThroughFooBars',
	fields=[
	('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
	('foo', models.ForeignKey(on_delete=models.deletion.CASCADE, to='through.Foo')),
	('bar', models.ForeignKey(on_delete=models.deletion.CASCADE, to='through.Bar')),
	],
	options={
	'db_table': 'through_foo_bars',
	'unique_together': (('foo', 'bar'),),
	},
	),
	migrations.AlterField(
	model_name='foo',
	name='bars',
	field=models.ManyToManyField(through='through.ThroughFooBars', to='through.Bar'),
	),
	]
	operations = [
	migrations.SeparateDatabaseAndState(state_operations=state_operations),
	migrations.AddField(
	model_name='throughfoobars',
	name='order',
	field=models.PositiveIntegerField(default=0),
	),
	migrations.AlterModelOptions(
	name='throughfoobars',
	options={'ordering': ('order',)},
	),
	]

view raw 0002_though.py hosted with ❤ by GitHub

We can check if the migration was succesful with inspectdb:

$ ./manage.py inspectdb through_foo_bars

	from django.db import models
	class ThroughFooBars(models.Model):
	foo = models.ForeignKey('ThroughFoo', models.DO_NOTHING)
	bar = models.ForeignKey('ThroughBar', models.DO_NOTHING)
	order = models.PositiveIntegerField()
	class Meta:
	managed = False
	db_table = 'through_foo_bars'
	unique_together = (('foo', 'bar'),)

view raw inspectdb.py hosted with ❤ by GitHub

Seems to look good to me?