Commit 3673f624 authored by Eliot Berriot's avatar Eliot Berriot

Merge branch 'release/0.7'

parents 6bf73384 b780bee8
......@@ -16,13 +16,15 @@ test_api:
stage: test
image: funkwhale/funkwhale:latest
cache:
key: "$CI_PROJECT_ID/pip_cache"
key: "$CI_PROJECT_ID__pip_cache"
paths:
- "$PIP_CACHE_DIR"
variables:
DJANGO_ALLOWED_HOSTS: "localhost"
DATABASE_URL: "postgresql://postgres@postgres/postgres"
FUNKWHALE_URL: "https://funkwhale.ci"
CACHEOPS_ENABLED: "false"
before_script:
- cd api
- pip install -r requirements/base.txt
......@@ -44,7 +46,7 @@ test_front:
- yarn install
- yarn run unit
cache:
key: "$CI_PROJECT_ID/front_dependencies"
key: "$CI_PROJECT_ID__front_dependencies"
paths:
- front/node_modules
- front/yarn.lock
......@@ -66,7 +68,7 @@ build_front:
- yarn install
- yarn run build
cache:
key: "$CI_PROJECT_ID/front_dependencies"
key: "$CI_PROJECT_ID__front_dependencies"
paths:
- front/node_modules
- front/yarn.lock
......@@ -84,15 +86,12 @@ build_front:
pages:
stage: test
image: alpine
image: python:3.6-alpine
before_script:
- cd docs
script:
- apk --no-cache add py2-pip python-dev
- pip install sphinx
- apk --no-cache add make
- make html
- mv _build/html/ ../public
- python -m sphinx . ../public
artifacts:
paths:
- public
......
......@@ -3,7 +3,37 @@ Changelog
.. towncrier
0.6.1 (unreleased)
0.7 (2018-03-21)
----------------
Features:
- Can now filter artists and albums with no listenable tracks (#114)
- Improve the style of the sidebar to make it easier to understand which tab is
selected (#118)
- On artist page, albums are not sorted by release date, if any (#116)
- Playlists are here \o/ :tada: (#3, #93, #94)
- Use django-cacheops to cache common ORM requests (#117)
Bugfixes:
- Fixed broken import request admin (#115)
- Fixed forced redirection to login event with
API_AUTHENTICATION_REQUIRED=False (#119)
- Fixed position not being reseted properly when playing the same track
multiple times in a row
- Fixed synchronized start/stop radio buttons for all custom radios (#103)
- Fixed typo and missing icon on homepage (#96)
Documentation:
- Up-to-date and complete development and contribution instructions in
README.rst (#123)
0.6.1 (2018-03-06)
------------------
Features:
......
......@@ -5,27 +5,107 @@ A self-hosted tribute to Grooveshark.com.
LICENSE: BSD
Setting up a development environment (docker)
----------------------------------------------
Getting help
------------
First of all, pull the repository.
We offer various Matrix.org rooms to discuss about funkwhale:
Then, pull and build all the containers::
- `#funkwhale:matrix.org <https://riot.im/app/#/room/#funkwhale:matrix.org>`_ for general questions about funkwhale
- `#funkwhale-dev:matrix.org <https://riot.im/app/#/room/#funkwhale-dev:matrix.org>`_ for development-focused discussion
Please join those rooms if you have any questions!
Running the development version
-------------------------------
If you want to fix a bug or implement a feature, you'll need
to run a local, development copy of funkwhale.
We provide a docker based development environment, which should
be both easy to setup and work similarly regardless of your
development machine setup.
Instructions for bare-metal setup will come in the future (Merge requests
are welcome).
Installing docker and docker-compose
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This is already cover in the relevant documentations:
- https://docs.docker.com/install/
- https://docs.docker.com/compose/install/
Cloning the project
^^^^^^^^^^^^^^^^^^^
Visit https://code.eliotberriot.com/funkwhale/funkwhale and clone the repository using SSH or HTTPS. Exemple using SSH::
git clone ssh://git@code.eliotberriot.com:2222/funkwhale/funkwhale.git
cd funkwhale
A note about branches
^^^^^^^^^^^^^^^^^^^^^
Next release development occurs on the "develop" branch, and releases are made on the "master" branch. Therefor, when submitting Merge Requests, ensure you are merging on the develop branch.
Working with docker
^^^^^^^^^^^^^^^^^^^
In developpement, we use the docker-compose file named ``dev.yml``, and this is why all our docker-compose commands will look like this::
docker-compose -f dev.yml logs
If you do not want to add the ``-f dev.yml`` snippet everytime, you can run this command before starting your work::
export COMPOSE_FILE=dev.yml
Building the containers
^^^^^^^^^^^^^^^^^^^^^^^
On your initial clone, or if there have been some changes in the
app dependencies, you will have to rebuild your containers. This is done
via the following command::
docker-compose -f dev.yml build
docker-compose -f dev.yml pull
API setup
^^^^^^^^^^
Database management
^^^^^^^^^^^^^^^^^^^
To setup funkwhale's database schema, run this::
docker-compose -f dev.yml run --rm api python manage.py migrate
This will create all the tables needed for the API to run proprely.
You will also need to run this whenever changes are made on the database
schema.
It is safe to run this command multiple times, so you can run it whenever
you fetch develop.
You'll have apply database migrations::
Development data
^^^^^^^^^^^^^^^^
docker-compose -f dev.yml run celeryworker python manage.py migrate
You'll need at least an admin user and some artists/tracks/albums to work
locally.
And to create an admin user::
Create an admin user with the following command::
docker-compose -f dev.yml run celeryworker python manage.py createsuperuser
docker-compose -f dev.yml run --rm api python manage.py createsuperuser
Injecting fake data is done by running the fllowing script::
artists=25
command="from funkwhale_api.music import fake_data; fake_data.create_data($artists)"
echo $command | docker-compose -f dev.yml run --rm api python manage.py shell -i python
The previous command will create 25 artists with random albums, tracks
and metadata.
Launch all services
......@@ -33,18 +113,83 @@ Launch all services
Then you can run everything with::
docker-compose up
docker-compose -f dev.yml up
This will launch all services, and output the logs in your current terminal window.
If you prefer to launch them in the background instead, use the ``-d`` flag, and access the logs when you need it via ``docker-compose -f dev.yml logs --tail=50 --follow``.
Once everything is up, you can access the various funkwhale's components:
- The Vue webapp, on http://localhost:8080
- The API, on http://localhost:8080/api/v1/
- The django admin, on http://localhost:8080/api/admin/
The API server will be accessible at http://localhost:6001, and the front-end at http://localhost:8080.
Running API tests
------------------
^^^^^^^^^^^^^^^^^
To run the pytest test suite, use the following command::
docker-compose -f dev.yml run --rm api pytest
This is regular pytest, so you can use any arguments/options that pytest usually accept::
# get some help
docker-compose -f dev.yml run --rm api pytest -h
# Stop on first failure
docker-compose -f dev.yml run --rm api pytest -x
# Run a specific test file
docker-compose -f dev.yml run --rm api pytest tests/test_acoustid.py
Running front-end tests
^^^^^^^^^^^^^^^^^^^^^^^
To run the front-end test suite, use the following command::
docker-compose -f dev.yml run --rm front yarn run unit
We also support a "watch and test" mode were we continually relaunch
tests when changes are recorded on the file system::
docker-compose -f dev.yml run --rm front yarn run unit-watch
The latter is especially useful when you are debugging failing tests.
.. note::
The front-end test suite coverage is still pretty low
Stopping everything
^^^^^^^^^^^^^^^^^^^
Once you're down with your work, you can stop running containers, if any, with::
docker-compose -f dev.yml stop
Removing everything
^^^^^^^^^^^^^^^^^^^
If you want to wipe your development environment completely (e.g. if you want to start over from scratch), just run::
docker-compose -f dev.yml down -v
This will wipe your containers and data, so please be careful before running it.
Everything is managed using docker and docker-compose, just run::
You can keep your data by removing the ``-v`` flag.
./api/runtests
This bash script invoke `python manage.py test` in a docker container under the hood, so you can use
traditional django test arguments and options, such as::
Typical workflow for a merge request
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
./api/runtests funkwhale_api.music # run a specific app test
0. Fork the project if you did not already or if you do not have access to the main repository
1. Checkout the development branch and pull most recent changes: ``git checkout develop && git pull``
2. Create a dedicated branch for your work ``42-awesome-fix``. It is good practice to prefix your branch name with the ID of the issue you are solving.
3. Work on your stuff
4. Commit small, atomic changes to make it easier to review your contribution
5. Add a changelog fragment to summarize your changes: ``echo "Implemented awesome stuff (#42)" > changes/changelog.d/42.feature"``
6. Push your branch
7. Create your merge request
8. Take a step back and enjoy, we're really grateful you did all of this and took the time to contribute!
#!/bin/bash
set -e
if [ $1 = "pytest" ]; then
# let pytest.ini handle it
unset DJANGO_SETTINGS_MODULE
fi
exec "$@"
......@@ -57,9 +57,9 @@ THIRD_PARTY_APPS = (
'taggit',
'rest_auth',
'rest_auth.registration',
'mptt',
'dynamic_preferences',
'django_filters',
'cacheops',
)
......@@ -369,7 +369,19 @@ MUSICBRAINZ_CACHE_DURATION = env.int(
'MUSICBRAINZ_CACHE_DURATION',
default=300
)
CACHEOPS_REDIS = env('CACHE_URL', default=CACHE_DEFAULT)
CACHEOPS_ENABLED = env.bool('CACHEOPS_ENABLED', default=True)
CACHEOPS = {
'music.artist': {'ops': 'all', 'timeout': 60 * 60},
'music.album': {'ops': 'all', 'timeout': 60 * 60},
'music.track': {'ops': 'all', 'timeout': 60 * 60},
'music.trackfile': {'ops': 'all', 'timeout': 60 * 60},
'taggit.tag': {'ops': 'all', 'timeout': 60 * 60},
}
# Custom Admin URL, use {% url 'admin:index' %}
ADMIN_URL = env('DJANGO_ADMIN_URL', default='^api/admin/')
CSRF_USE_SESSIONS = True
# Playlist settings
PLAYLISTS_MAX_TRACKS = env.int('PLAYLISTS_MAX_TRACKS', default=250)
......@@ -19,10 +19,6 @@ CACHES = {
CELERY_BROKER_URL = 'memory://'
# TESTING
# ------------------------------------------------------------------------------
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
########## CELERY
# In development, all tasks will be executed locally by blocking until the task returns
CELERY_TASK_ALWAYS_EAGER = True
......@@ -30,3 +26,4 @@ CELERY_TASK_ALWAYS_EAGER = True
# Your local stuff: Below this line define 3rd party library settings
API_AUTHENTICATION_REQUIRED = False
CACHEOPS_ENABLED = False
version: '2'
services:
postgres:
image: postgres:9.5
api:
build: .
links:
- postgres
- redis
command: ./compose/django/gunicorn.sh
env_file: .env
volumes:
- ./media:/app/funkwhale_api/media
- ./staticfiles:/app/staticfiles
- ./music:/music
ports:
- "127.0.0.1:6001:5000"
redis:
image: redis:3.0
celeryworker:
build: .
env_file: .env
links:
- postgres
- redis
command: celery -A funkwhale_api.taskapp worker -l INFO
volumes:
- ./media:/app/funkwhale_api/media
- ./music:/music
environment:
- C_FORCE_ROOT=True
celerybeat:
build: .
env_file: .env
links:
- postgres
- redis
command: celery -A funkwhale_api.taskapp beat -l INFO
......@@ -23,3 +23,4 @@ RUN pip install -r /requirements/test.txt
COPY . /app
WORKDIR /app
ENTRYPOINT ["compose/django/dev-entrypoint.sh"]
# -*- coding: utf-8 -*-
__version__ = '0.6.1'
__version__ = '0.7'
__version_info__ = tuple([int(num) if num.isdigit() else num for num in __version__.replace('-', '.', 1).split('.')])
from django.db import models
PRIVACY_LEVEL_CHOICES = [
('me', 'Only me'),
('followers', 'Me and my followers'),
('instance', 'Everyone on my instance, and my followers'),
('everyone', 'Everyone, including people on other instances'),
]
def get_privacy_field():
return models.CharField(
max_length=30, choices=PRIVACY_LEVEL_CHOICES, default='instance')
def privacy_level_query(user, lookup_field='privacy_level'):
if user.is_anonymous:
return models.Q(**{
lookup_field: 'everyone',
})
return models.Q(**{
'{}__in'.format(lookup_field): [
'me', 'followers', 'instance', 'everyone'
]
})
import operator
from django.conf import settings
from django.http import Http404
from rest_framework.permissions import BasePermission, DjangoModelPermissions
......@@ -20,3 +23,39 @@ class HasModelPermission(DjangoModelPermissions):
"""
def get_required_permissions(self, method, model_cls):
return super().get_required_permissions(method, self.model)
class OwnerPermission(BasePermission):
"""
Ensure the request user is the owner of the object.
Usage:
class MyView(APIView):
model = MyModel
permission_classes = [OwnerPermission]
owner_field = 'owner'
owner_checks = ['read', 'write']
"""
perms_map = {
'GET': 'read',
'OPTIONS': 'read',
'HEAD': 'read',
'POST': 'write',
'PUT': 'write',
'PATCH': 'write',
'DELETE': 'write',
}
def has_object_permission(self, request, view, obj):
method_check = self.perms_map[request.method]
owner_checks = getattr(view, 'owner_checks', ['read', 'write'])
if method_check not in owner_checks:
# check not enabled
return True
owner_field = getattr(view, 'owner_field', 'user')
owner = operator.attrgetter(owner_field)(obj)
if owner != request.user:
raise Http404
return True
import django_filters
from django.db.models import Count
from django_filters import rest_framework as filters
from . import models
class ArtistFilter(django_filters.FilterSet):
class ListenableMixin(filters.FilterSet):
listenable = filters.BooleanFilter(name='_', method='filter_listenable')
def filter_listenable(self, queryset, name, value):
queryset = queryset.annotate(
files_count=Count('tracks__files')
)
if value:
return queryset.filter(files_count__gt=0)
else:
return queryset.filter(files_count=0)
class ArtistFilter(ListenableMixin):
class Meta:
model = models.Artist
fields = {
'name': ['exact', 'iexact', 'startswith', 'icontains']
'name': ['exact', 'iexact', 'startswith', 'icontains'],
'listenable': 'exact',
}
class AlbumFilter(ListenableMixin):
listenable = filters.BooleanFilter(name='_', method='filter_listenable')
class Meta:
model = models.Album
fields = ['listenable']
......@@ -54,6 +54,7 @@ class TagViewSetMixin(object):
queryset = queryset.filter(tags__pk=tag)
return queryset
class ArtistViewSet(SearchMixin, viewsets.ReadOnlyModelViewSet):
queryset = (
models.Artist.objects.all()
......@@ -67,6 +68,7 @@ class ArtistViewSet(SearchMixin, viewsets.ReadOnlyModelViewSet):
filter_class = filters.ArtistFilter
ordering_fields = ('id', 'name', 'creation_date')
class AlbumViewSet(SearchMixin, viewsets.ReadOnlyModelViewSet):
queryset = (
models.Album.objects.all()
......@@ -78,6 +80,7 @@ class AlbumViewSet(SearchMixin, viewsets.ReadOnlyModelViewSet):
permission_classes = [ConditionalAuthentication]
search_fields = ['title__unaccent']
ordering_fields = ('creation_date',)
filter_class = filters.AlbumFilter
class ImportBatchViewSet(
......@@ -237,6 +240,7 @@ class TagViewSet(viewsets.ReadOnlyModelViewSet):
class Search(views.APIView):
max_results = 3
permission_classes = [ConditionalAuthentication]
def get(self, request, *args, **kwargs):
query = request.GET['query']
......
......@@ -5,13 +5,13 @@ from . import models
@admin.register(models.Playlist)
class PlaylistAdmin(admin.ModelAdmin):
list_display = ['name', 'user', 'is_public', 'creation_date']
list_display = ['name', 'user', 'privacy_level', 'creation_date']
search_fields = ['name', ]
list_select_related = True
@admin.register(models.PlaylistTrack)
class PlaylistTrackAdmin(admin.ModelAdmin):
list_display = ['playlist', 'track', 'position', ]
list_display = ['playlist', 'track', 'index']
search_fields = ['track__name', 'playlist__name']
list_select_related = True
import factory
from funkwhale_api.factories import registry
from funkwhale_api.music.factories import TrackFactory
from funkwhale_api.users.factories import UserFactory
......@@ -11,3 +12,12 @@ class PlaylistFactory(factory.django.DjangoModelFactory):
class Meta:
model = 'playlists.Playlist'
@registry.register
class PlaylistTrackFactory(factory.django.DjangoModelFactory):
playlist = factory.SubFactory(PlaylistFactory)
track = factory.SubFactory(TrackFactory)
class Meta:
model = 'playlists.PlaylistTrack'
from django_filters import rest_framework as filters
from funkwhale_api.music import utils
from . import models
class PlaylistFilter(filters.FilterSet):
q = filters.CharFilter(name='_', method='filter_q')
class Meta:
model = models.Playlist
fields = {
'user': ['exact'],
'name': ['exact', 'icontains'],
'q': 'exact',
}
def filter_q(self, queryset, name, value):
query = utils.get_query(value, ['name', 'user__username'])
return queryset.filter(query)
......@@ -4,7 +4,6 @@ from __future__ import unicode_literals
from django.db import migrations, models
from django.conf import settings
import django.utils.timezone
import mptt.fields
class Migration(migrations.Migration):
......@@ -34,7 +33,7 @@ class Migration(migrations.Migration):
('tree_id', models.PositiveIntegerField(db_index=True, editable=False)),
('position', models.PositiveIntegerField(db_index=True, editable=False)),
('playlist', models.ForeignKey(to='playlists.Playlist', related_name='playlist_tracks', on_delete=models.CASCADE)),
('previous', mptt.fields.TreeOneToOneField(null=True, to='playlists.PlaylistTrack', related_name='next', blank=True, on_delete=models.CASCADE)),
('previous', models.OneToOneField(null=True, to='playlists.PlaylistTrack', related_name='next', blank=True, on_delete=models.CASCADE)),
('track', models.ForeignKey(to='music.Track', related_name='playlist_tracks', on_delete=models.CASCADE)),
],
options={
......
# Generated by Django 2.0.3 on 2018-03-16 22:17
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('playlists', '0001_initial'),
]
operations = [
migrations.RemoveField(
model_name='playlist',
name='is_public',
),
migrations.AddField(
model_name='playlist',
name='privacy_level',
field=models.CharField(choices=[('me', 'Only me'), ('followers', 'Me and my followers'), ('instance', 'Everyone on my instance, and my followers'), ('everyone', 'Everyone, including people on other instances')], default='instance', max_length=30),
),
]
# Generated by Django 2.0.3 on 2018-03-19 12:14
from django.db import migrations, models
import django.utils.timezone
class Migration(migrations.Migration):
dependencies = [
('playlists', '0002_auto_20180316_2217'),
]
operations = [
migrations.AlterModelOptions(
name='playlisttrack',
options={'ordering': ('-playlist', 'index')},
),
migrations.AddField(
model_name='playlisttrack',
name='creation_date',
field=models.DateTimeField(default=django.utils.timezone.now),
),
migrations.AddField(
model_name='playlisttrack',
name='index',
field=models.PositiveIntegerField(null=True),
),
migrations.RemoveField(
model_name='playlisttrack',
name='lft',
),
migrations.RemoveField(
model_name='playlisttrack',
name='position',
),
migrations.RemoveField(
model_name='playlisttrack',
name='previous',
),
migrations.RemoveField(
model_name='playlisttrack',
name='rght',
),
migrations.RemoveField(
model_name='playlisttrack',
name='tree_id',
),
migrations.AlterUniqueTogether(
name='playlisttrack',
unique_together={('playlist', 'index')},
),
]
# Generated by Django 2.0.3 on 2018-03-20 17:13
from django.db import migrations, models
class Migration(migrations.Migration):