1 Commits

Author SHA1 Message Date
db4ad6ec40 WIP instructions for dev installation from scratch 2017-02-09 23:24:12 +01:00
173 changed files with 610 additions and 15499 deletions

13
.gitignore vendored
View File

@@ -3,12 +3,6 @@
.coverage .coverage
*.pyc *.pyc
__pycache__ __pycache__
*.js.map
*.css.map
/cloud/templates/
/cloud/static/assets/
node_modules/
/config_local.py /config_local.py
@@ -18,9 +12,4 @@ node_modules/
/.eggs/ /.eggs/
/dump/ /dump/
/google_app*.json /google_app*.json
/docker/2_buildpy/python/ /docker/3_run/wheelhouse/
/docker/4_run/wheelhouse/
/docker/4_run/deploy/
/celerybeat-schedule.bak
/celerybeat-schedule.dat
/celerybeat-schedule.dir

162
README.md
View File

@@ -1,97 +1,43 @@
# Blender Cloud # Blender Cloud
Welcome to the [Blender Cloud](https://cloud.blender.org/) code repo! Welcome to the [Blender Cloud](https://cloud.blender.org) code repo!
Blender Cloud runs on the [Pillar](https://pillarframework.org/) framework. Blender Cloud runs on the [Pillar](https://pillarframework.org) framework.
## Development setup ## Quick setup
Jumpstart Blender Cloud development with this simple guide. Set up a node with these commands. Note that that script is already outdated...
### System setup
Blender Cloud relies on a number of services in order to run. Check out the [Pillar system setup](
https://pillarframework.org/development/system_setup/#step-by-step-setup) to set this up.
Add `cloud.local` to the `/etc/hosts` file on localhost. This is a development convention.
### Check out the code
Go to the local development directory and check out the following repositories, next to each other.
``` ```
cd /home/guest/Developer #!/usr/bin/env bash
mkdir -p /data/git
mkdir -p /data/storage
mkdir -p /data/config
mkdir -p /data/certs
sudo apt-get update
sudo apt-get -y install python-pip
pip install docker-compose
cd /data/git
git clone git://git.blender.org/pillar-python-sdk.git git clone git://git.blender.org/pillar-python-sdk.git
git clone git://git.blender.org/pillar.git git clone git://git.blender.org/pillar-server.git pillar
git clone git://git.blender.org/attract.git git clone git://git.blender.org/attract.git
git clone git://git.blender.org/flamenco.git git clone git://git.blender.org/flamenco.git
git clone git://git.blender.org/pillar-svnman.git
git clone git://git.blender.org/blender-cloud.git git clone git://git.blender.org/blender-cloud.git
``` ```
### Initial setup and configuration ## Deploying to production server
Create a virtualenv for the project and install the requirements: First of all, add those aliases to the `[alias]` section of your `~/.gitconfig`
```
cd blender-cloud
mkvirtualenv blender-cloud -p python3.6
pip install -r requirements-dev.txt
```
Build assets and templates for all Blender Cloud dependencies using Gulp.
```
./gulp all
```
Make a copy of the config_local example, which will be further edited as the application is
configured.
```
cp config_local.example.py config_local.py
```
Setup the database with the initial collections and the admin user.
```
./manage.py setup setup_db your_email
```
The command will return the following message:
```
Created project <project_id> for user <user_id>
```
Copy the value of `<project_id>` and assign it as value for `MAIN_PROJECT_ID`.
Run the application:
```
./manage.py runserver
```
## Development
When ready to commit, change the remotes to work over SSH. For example:
`git remote set-url origin git@git.blender.org:blender-cloud.git`
For more information, check out [this guide](https://wiki.blender.org/wiki/Tools/Git#Commit_Access).
## Preparing the production branch for deployment
All revisions to deploy to production should be on the `production` branches of all the relevant
repositories.
Make sure you have these aliases in the `[alias]` section of your `~/.gitconfig`:
``` ```
prod = "!git checkout production && git fetch origin production && gitk --all" prod = "!git checkout production && git fetch origin production && gitk --all"
ff = "merge --ff-only" ff = "merge --ff-only"
pp = "!git push && if [ -e deploy.sh ]; then ./deploy.sh; fi && git checkout master"
``` ```
The following commands should be executed for each (sub)project; specifically for The following commands should be executed for each subproject; specifically for
the current repository, Pillar, Attract, Flamenco, and Pillar-SVNMan: Pillar and Attract:
``` ```
cd $projectdir cd $projectdir
@@ -114,19 +60,59 @@ git ff master
# Run tests again # Run tests again
py.test py.test
# Push the production branch. # Push the production branch and run dummy deploy script.
git push git pp # pp = "Push to Production"
# The above alias waits for [ENTER] until all deploys are done.
# Let it wait, perform the other commands in another terminal.
``` ```
## Deploying to production server Now follow the above receipe on the Blender Cloud project as well.
Contrary to the subprojects, `git pp` will actually perform the deploy
for real.
``` Now you can press `[ENTER]` in the Pillar and Attract terminals that
workon blender-cloud # activate your virtualenv were still waiting for it.
cd $projectdir/deploy
./2docker.sh
./build-all.sh # or ./build-quick.sh
./2server.sh servername
```
To deploy another branch than `production`, do `export DEPLOY_BRANCH=otherbranch` before starting After everything is done, your (sub)projects should all be back on
the above commands. the master branch.
## Updating dependencies via Docker images
To update dependencies that need compiling, you need the `2_build` docker
container. To rebuild the lot, run `docker/build.sh`.
Follow these steps to deploy the new container on production:
1. run `docker/build.sh`
2. `docker push armadillica/blender_cloud`
On the production machine:
1. `docker pull armadillica/blender_cloud`
2. `docker-compose up -d` (from the `/data/git/blender-cloud/docker` directory)
## Development setup
Here are some notes on how to get the whole Blender Cloud dev environment set up
from scratch. First off, we agree to develop on bare metal and use Docker only for
production.
Initial requirements:
- Python 2.7
- NPM
- Blender ID server up and running (see blender-id for more)
Here are the steps:
- Clone the fantastic 5 (pillar, pillar-python-sdk, attract, flamenco and
blender-cloud)
- Run `npm install` in pillar, attract and flamenco
- Create `config_local.py` in blender-cloud
- `pip install -e .` in every repo except blender-cloud
- `pip install -r requirements.txt` in blender-cloud
- setup_db
- create_urler_account

View File

@@ -3,19 +3,13 @@
from pillar import PillarServer from pillar import PillarServer
from attract import AttractExtension from attract import AttractExtension
from flamenco import FlamencoExtension from flamenco import FlamencoExtension
from svnman import SVNManExtension
from cloud import CloudExtension
attract = AttractExtension() attract = AttractExtension()
flamenco = FlamencoExtension() flamenco = FlamencoExtension()
svnman = SVNManExtension()
cloud = CloudExtension()
app = PillarServer('.') app = PillarServer('.')
app.load_extension(attract, '/attract') app.load_extension(attract, '/attract')
app.load_extension(flamenco, '/flamenco') app.load_extension(flamenco, '/flamenco')
app.load_extension(svnman, '/svn')
app.load_extension(cloud, None)
app.process_extensions() app.process_extensions()
if __name__ == '__main__': if __name__ == '__main__':

View File

@@ -1,103 +0,0 @@
import logging
import flask
from werkzeug.local import LocalProxy
from pillar.api.utils import authorization
from pillar.extension import PillarExtension
EXTENSION_NAME = 'cloud'
class CloudExtension(PillarExtension):
has_context_processor = True
user_roles = {'subscriber-pro', 'has_subscription'}
user_roles_indexable = {'subscriber-pro', 'has_subscription'}
user_caps = {
'has_subscription': {'can-renew-subscription'},
}
def __init__(self):
self._log = logging.getLogger('%s.CloudExtension' % __name__)
@property
def name(self):
return EXTENSION_NAME
def flask_config(self):
"""Returns extension-specific defaults for the Flask configuration.
Use this to set sensible default values for configuration settings
introduced by the extension.
:rtype: dict
"""
# Just so that it registers the management commands.
from . import cli
return {
'EXTERNAL_SUBSCRIPTIONS_MANAGEMENT_SERVER': 'https://store.blender.org/api/',
'EXTERNAL_SUBSCRIPTIONS_TIMEOUT_SECS': 10,
'BLENDER_ID_WEBHOOK_USER_CHANGED_SECRET': 'oos9wah1Zoa0Yau6ahThohleiChephoi',
'NODE_TAGS': ['animation', 'modelling', 'rigging'],
}
def eve_settings(self):
"""Returns extensions to the Eve settings.
Currently only the DOMAIN key is used to insert new resources into
Eve's configuration.
:rtype: dict
"""
return {}
def blueprints(self):
"""Returns the list of top-level blueprints for the extension.
These blueprints will be mounted at the url prefix given to
app.load_extension().
:rtype: list of flask.Blueprint objects.
"""
from . import routes
import cloud.stats.routes
return [
routes.blueprint,
cloud.stats.routes.blueprint,
]
@property
def template_path(self):
import os.path
return os.path.join(os.path.dirname(__file__), 'templates')
@property
def static_path(self):
import os.path
return os.path.join(os.path.dirname(__file__), 'static')
def context_processor(self):
return {
'current_user_is_subscriber': authorization.user_has_cap('subscriber')
}
def setup_app(self, app):
from . import routes, webhooks, eve_hooks, email
routes.setup_app(app)
app.register_api_blueprint(webhooks.blueprint, '/webhooks')
eve_hooks.setup_app(app)
email.setup_app(app)
def _get_current_cloud():
"""Returns the Cloud extension of the current application."""
return flask.current_app.pillar_extensions[EXTENSION_NAME]
current_cloud = LocalProxy(_get_current_cloud)
"""Cloud extension of the current app."""

View File

@@ -1,129 +0,0 @@
#!/usr/bin/env python
import logging
from urllib.parse import urljoin
from flask import current_app
from flask_script import Manager
import requests
from pillar.cli import manager
from pillar.api import service
log = logging.getLogger(__name__)
manager_cloud = Manager(
current_app, usage="Blender Cloud scripts")
@manager_cloud.command
def create_groups():
"""Creates the admin/demo/subscriber groups."""
import pprint
group_ids = {}
groups_coll = current_app.db('groups')
for group_name in ['admin', 'demo', 'subscriber']:
if groups_coll.find({'name': group_name}).count():
log.info('Group %s already exists, skipping', group_name)
continue
result = groups_coll.insert_one({'name': group_name})
group_ids[group_name] = result.inserted_id
service.fetch_role_to_group_id_map()
log.info('Created groups:\n%s', pprint.pformat(group_ids))
@manager_cloud.command
def reconcile_subscribers():
"""For every user, check their subscription status with the store."""
import threading
import concurrent.futures
from pillar.auth import UserClass
from pillar.api import blender_id
from pillar.api.blender_cloud.subscription import do_update_subscription
sessions = threading.local()
service.fetch_role_to_group_id_map()
users_coll = current_app.data.driver.db['users']
found = users_coll.find({'auth.provider': 'blender-id'})
count_users = found.count()
count_skipped = count_processed = 0
log.info('Processing %i users', count_users)
lock = threading.Lock()
real_current_app = current_app._get_current_object()
api_token = real_current_app.config['BLENDER_ID_USER_INFO_TOKEN']
api_url = real_current_app.config['BLENDER_ID_USER_INFO_API']
def do_user(idx, user):
nonlocal count_skipped, count_processed
log.info('Processing %i/%i %s', idx + 1, count_users, user['email'])
# Get the Requests session for this thread.
try:
sess = sessions.session
except AttributeError:
sess = sessions.session = requests.Session()
# Get the info from Blender ID
bid_user_id = blender_id.get_user_blenderid(user)
if not bid_user_id:
with lock:
count_skipped += 1
return
url = urljoin(api_url, bid_user_id)
resp = sess.get(url, headers={'Authorization': f'Bearer {api_token}'})
if resp.status_code == 404:
log.info('User %s with Blender ID %s not found, skipping', user['email'], bid_user_id)
with lock:
count_skipped += 1
return
if resp.status_code != 200:
log.error('Unable to reach Blender ID (code %d), aborting', resp.status_code)
with lock:
count_skipped += 1
return
bid_user = resp.json()
if not bid_user:
log.error('Unable to parse response for user %s, aborting', user['email'])
with lock:
count_skipped += 1
return
# Actually update the user, and do it thread-safe just to be sure.
with real_current_app.app_context():
local_user = UserClass.construct('', user)
with lock:
do_update_subscription(local_user, bid_user)
count_processed += 1
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
future_to_user = {executor.submit(do_user, idx, user): user
for idx, user in enumerate(found)}
for future in concurrent.futures.as_completed(future_to_user):
user = future_to_user[future]
try:
future.result()
except Exception as ex:
log.exception('Error updating user %s', user)
log.info('Done reconciling %d subscribers', count_users)
log.info(' processed: %d', count_processed)
log.info(' skipped : %d', count_skipped)
manager.add_command("cloud", manager_cloud)

View File

@@ -1,33 +0,0 @@
import functools
import logging
import flask
from pillar.auth import UserClass
log = logging.getLogger(__name__)
def queue_welcome_mail(user: UserClass):
"""Queue a welcome email for execution by Celery."""
assert user.email
log.info('queueing welcome email to %s', user.email)
subject = 'Welcome to Blender Cloud'
render = functools.partial(flask.render_template, subject=subject, user=user)
text = render('emails/welcome.txt')
html = render('emails/welcome.html')
from pillar.celery import email_tasks
email_tasks.send_email.delay(user.full_name, user.email, subject, text, html)
def user_subscription_changed(user: UserClass, *, grant_roles: set, revoke_roles: set):
if user.has_cap('subscriber') and 'has_subscription' in grant_roles:
log.info('user %s just got a new subscription', user.email)
queue_welcome_mail(user)
def setup_app(app):
from pillar.api.blender_cloud import subscription
subscription.user_subscription_updated.connect(user_subscription_changed)

View File

@@ -1,39 +0,0 @@
import logging
import typing
from pillar.auth import UserClass
from . import email
log = logging.getLogger(__name__)
def welcome_new_user(user_doc: dict):
"""Sends a welcome email to a new user."""
user_email = user_doc.get('email')
if not user_email:
log.warning('user %s has no email address', user_doc.get('_id', '-no-id-'))
return
# Only send mail to new users when they actually are subscribers.
user = UserClass.construct('', user_doc)
if not (user.has_cap('subscriber') or user.has_cap('can-renew-subscription')):
log.debug('user %s is new, but not a subscriber, so no email for them.', user_email)
return
email.queue_welcome_mail(user)
def welcome_new_users(user_docs: typing.List[dict]):
"""Sends a welcome email to new users."""
for user_doc in user_docs:
try:
welcome_new_user(user_doc)
except Exception:
log.exception('error sending welcome mail to user %s', user_doc)
def setup_app(app):
app.on_inserted_users += welcome_new_users

View File

@@ -1,473 +0,0 @@
import functools
import json
import logging
import typing
from flask_login import current_user, login_required
import flask
from flask import Blueprint, render_template, redirect, session, url_for, abort, flash
from pillarsdk import Node, Project, User, exceptions as sdk_exceptions, Group
from pillarsdk.exceptions import ResourceNotFound
from pillar import current_app
import pillar.api
from pillar.web.users import forms
from pillar.web.utils import system_util, get_file, current_user_is_authenticated
from pillar.web.utils import attach_project_pictures
from pillar.web.settings import blueprint as blueprint_settings
from pillar.web.nodes.routes import url_for_node
from pillar.web.nodes.custom.comments import render_comments_for_node
from pillar.web.projects.routes import render_project
from pillar.web.projects.routes import find_project_or_404
blueprint = Blueprint('cloud', __name__)
log = logging.getLogger(__name__)
@blueprint.route('/')
def homepage():
if current_user.is_anonymous:
return redirect(url_for('cloud.welcome'))
return render_template(
'homepage.html',
api=system_util.pillar_api(),
**_homepage_context(),
)
def _homepage_context() -> dict:
"""Returns homepage template context variables."""
# Get latest blog posts
api = system_util.pillar_api()
latest_posts = Node.all({
'projection': {
'name': 1,
'project': 1,
'node_type': 1,
'picture': 1,
'properties.url': 1,
'properties.content': 1,
'properties.attachments': 1
},
'where': {'node_type': 'post', 'properties.status': 'published'},
'embedded': {'project': 1},
'sort': '-_created',
'max_results': '3'
}, api=api)
# Append picture Files to last_posts
for post in latest_posts._items:
post.picture = get_file(post.picture, api=api)
post.url = url_for_node(node=post)
# Get latest assets added to any project
latest_assets = Node.latest('assets', api=api)
# Append picture Files to latest_assets
for asset in latest_assets._items:
asset.picture = get_file(asset.picture, api=api)
asset.url = url_for_node(node=asset)
# Get latest comments to any node
latest_comments = Node.latest('comments', api=api)
# Get a list of random featured assets
random_featured = get_random_featured_nodes()
# Parse results for replies
to_remove = []
@functools.lru_cache()
def _find_parent(parent_node_id) -> Node:
return Node.find(parent_node_id,
{'projection': {
'_id': 1,
'name': 1,
'node_type': 1,
'project': 1,
'properties.url': 1,
}},
api=api)
for idx, comment in enumerate(latest_comments._items):
if comment.properties.is_reply:
try:
comment.attached_to = _find_parent(comment.parent.parent)
except ResourceNotFound:
# Remove this comment
to_remove.append(idx)
else:
comment.attached_to = comment.parent
for idx in reversed(to_remove):
del latest_comments._items[idx]
for comment in latest_comments._items:
if not comment.attached_to:
continue
comment.attached_to.url = url_for_node(node=comment.attached_to)
comment.url = url_for_node(node=comment)
main_project = Project.find(current_app.config['MAIN_PROJECT_ID'], api=api)
main_project.picture_header = get_file(main_project.picture_header, api=api)
# Merge latest assets and comments into one activity stream.
def sort_key(item):
return item._created
activity_stream = sorted(latest_assets._items, key=sort_key, reverse=True)
for node in activity_stream:
node.url = url_for_node(node=node)
return dict(
main_project=main_project,
latest_posts=latest_posts._items,
latest_comments=latest_comments._items,
activity_stream=activity_stream,
random_featured=random_featured)
@blueprint.route('/login')
def login():
from flask import request
if request.args.get('force'):
log.debug('Forcing logout of user before rendering login page.')
pillar.auth.logout_user()
next_after_login = request.args.get('next')
if not next_after_login:
next_after_login = request.referrer
session['next_after_login'] = next_after_login
return redirect(url_for('users.oauth_authorize', provider='blender-id'))
@blueprint.route('/welcome')
def welcome():
# Workaround to cache rendering of a page if user not logged in
@current_app.cache.cached(timeout=3600, unless=current_user_is_authenticated)
def render_page():
return render_template('welcome.html')
return render_page()
@blueprint.route('/about')
def about():
return render_template('about.html')
@blueprint.route('/services')
def services():
return render_template('services.html')
@blueprint.route('/stats')
def stats():
return render_template('stats.html')
@blueprint.route('/join')
def join():
"""Join page"""
return redirect('https://store.blender.org/product/membership/')
@blueprint.route('/renew')
def renew_subscription():
return render_template('renew_subscription.html')
def get_projects(category):
"""Utility to get projects based on category. Should be moved on the API
and improved with more extensive filtering capabilities.
"""
api = system_util.pillar_api()
projects = Project.all({
'where': {
'category': category,
'is_private': False},
'sort': '-_created',
}, api=api)
for project in projects._items:
attach_project_pictures(project, api)
return projects
@blueprint.route('/courses')
def courses():
@current_app.cache.cached(timeout=3600, unless=current_user_is_authenticated)
def render_page():
projects = get_projects('course')
return render_template(
'projects_index_collection.html',
title='courses',
projects=projects._items,
api=system_util.pillar_api())
return render_page()
@blueprint.route('/open-projects')
def open_projects():
@current_app.cache.cached(timeout=3600, unless=current_user_is_authenticated)
def render_page():
projects = get_projects('film')
return render_template(
'projects_index_collection.html',
title='open-projects',
projects=projects._items,
api=system_util.pillar_api())
return render_page()
@blueprint.route('/workshops')
def workshops():
@current_app.cache.cached(timeout=3600, unless=current_user_is_authenticated)
def render_page():
projects = get_projects('workshop')
return render_template(
'projects_index_collection.html',
title='workshops',
projects=projects._items,
api=system_util.pillar_api())
return render_page()
def get_random_featured_nodes() -> typing.List[dict]:
"""Returns a list of project/node combinations for featured nodes.
A random subset of 3 featured nodes from all public projects is returned.
Assumes that the user actually has access to the public projects' nodes.
The dict is a node, with a 'project' key that contains a projected project.
"""
proj_coll = current_app.db('projects')
featured_nodes = proj_coll.aggregate([
{'$match': {'is_private': False}},
{'$project': {'nodes_featured': True,
'url': True,
'name': True,
'summary': True,
'picture_square': True}},
{'$unwind': {'path': '$nodes_featured'}},
{'$sample': {'size': 3}},
{'$lookup': {'from': 'nodes',
'localField': 'nodes_featured',
'foreignField': '_id',
'as': 'node'}},
{'$unwind': {'path': '$node'}},
{'$project': {'url': True,
'name': True,
'summary': True,
'picture_square': True,
'node._id': True,
'node.name': True,
'node.permissions': True,
'node.picture': True,
'node.properties.content_type': True,
'node.properties.url': True}},
])
featured_node_documents = []
api = system_util.pillar_api()
for node_info in featured_nodes:
# Turn the project-with-node doc into a node-with-project doc.
node_document = node_info.pop('node')
node_document['project'] = node_info
node = Node(node_document)
node.picture = get_file(node.picture, api=api)
node.url = url_for_node(node=node)
node.project.url = url_for('projects.view', project_url=node.project.url)
node.project.picture_square = get_file(node.project.picture_square, api=api)
featured_node_documents.append(node)
return featured_node_documents
@blueprint_settings.route('/emails', methods=['GET', 'POST'])
@login_required
def emails():
"""Main email settings.
"""
if current_user.has_role('protected'):
return abort(404) # TODO: make this 403, handle template properly
api = system_util.pillar_api()
user = User.find(current_user.objectid, api=api)
# Force creation of settings for the user (safely remove this code once
# implemented on account creation level, and after adding settings to all
# existing users)
if not user.settings:
user.settings = dict(email_communications=1)
user.update(api=api)
if user.settings.email_communications is None:
user.settings.email_communications = 1
user.update(api=api)
# Generate form
form = forms.UserSettingsEmailsForm(
email_communications=user.settings.email_communications)
if form.validate_on_submit():
try:
user.settings.email_communications = form.email_communications.data
user.update(api=api)
flash("Profile updated", 'success')
except sdk_exceptions.ResourceInvalid as e:
message = json.loads(e.content)
flash(message)
return render_template('users/settings/emails.html', form=form, title='emails')
@blueprint_settings.route('/billing')
@login_required
def billing():
"""View the subscription status of a user
"""
from . import store
log.debug('START OF REQUEST')
if current_user.has_role('protected'):
return abort(404) # TODO: make this 403, handle template properly
expiration_date = 'No subscription to expire'
# Classify the user based on their roles and capabilities
cap_subs = current_user.has_cap('subscriber')
if current_user.has_role('demo'):
user_cls = 'demo'
elif not cap_subs and current_user.has_cap('can-renew-subscription'):
# This user has an inactive but renewable subscription.
user_cls = 'subscriber-expired'
elif cap_subs:
if current_user.has_role('subscriber'):
# This user pays for their own subscription. Only in this case do we need to fetch
# the expiration date from the Store.
user_cls = 'subscriber'
store_user = store.fetch_subscription_info(current_user.email)
if store_user is None:
expiration_date = 'Unable to reach Blender Store to check'
else:
expiration_date = store_user['expiration_date'][:10]
elif current_user.has_role('org-subscriber'):
# An organisation pays for this subscription.
user_cls = 'subscriber-org'
else:
# This user gets the subscription cap from somewhere else (like an organisation).
user_cls = 'subscriber-other'
else:
user_cls = 'outsider'
return render_template(
'users/settings/billing.html',
user_cls=user_cls,
expiration_date=expiration_date,
title='billing')
@blueprint.route('/terms-and-conditions')
def terms_and_conditions():
return render_template('terms_and_conditions.html')
@blueprint.route('/privacy')
def privacy():
return render_template('privacy.html')
@blueprint.route('/production')
def production():
return render_template('production.html')
@blueprint.route('/emails/welcome.send')
@login_required
def emails_welcome_send():
from cloud import email
email.queue_welcome_mail(current_user)
return f'queued mail to {current_user.email}'
@blueprint.route('/emails/welcome.html')
@login_required
def emails_welcome_html():
return render_template('emails/welcome.html',
subject='Welcome to Blender Cloud',
user=current_user)
@blueprint.route('/emails/welcome.txt')
@login_required
def emails_welcome_txt():
txt = render_template('emails/welcome.txt',
subject='Welcome to Blender Cloud',
user=current_user)
return flask.Response(txt, content_type='text/plain; charset=utf-8')
@blueprint.route('/nodes/<string(length=24):node_id>/comments')
def comments_for_node(node_id):
"""Overrides the default render_comments_for_node.
This is done in order to extend can_post_comments by requiring the
subscriber capability.
"""
api = system_util.pillar_api()
node = Node.find(node_id, api=api)
project = Project({'_id': node.project})
can_post_comments = project.node_type_has_method('comment', 'POST', api=api)
can_comment_override = flask.request.args.get('can_comment', 'True') == 'True'
can_post_comments = can_post_comments and can_comment_override and current_user.has_cap(
'subscriber')
return render_comments_for_node(node_id, can_post_comments=can_post_comments)
@blueprint.route('/p/hero')
def project_hero():
api = system_util.pillar_api()
project = find_project_or_404('hero',
embedded={'header_node': 1},
api=api)
# Load the header video file, if there is any.
header_video_file = None
header_video_node = None
if project.header_node and project.header_node.node_type == 'asset' and \
project.header_node.properties.content_type == 'video':
header_video_node = project.header_node
header_video_file = get_file(project.header_node.properties.file)
header_video_node.picture = get_file(header_video_node.picture)
pages = Node.all({
'where': {'project': project._id, 'node_type': 'page'},
'projection': {'name': 1}}, api=api)
return render_project(project, api,
extra_context={'header_video_file': header_video_file,
'header_video_node': header_video_node,
'pages': pages._items,},
template_name='projects/landing.html')
def setup_app(app):
global _homepage_context
cached = app.cache.cached(timeout=300)
_homepage_context = cached(_homepage_context)

Binary file not shown.

Before

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 68 KiB

View File

@@ -1,89 +0,0 @@
"""Interesting usage metrics"""
from flask import current_app
def count_nodes(query=None) -> int:
pipeline = [
{'$match': {'_deleted': {'$ne': 'true'}}},
{
'$lookup':
{
'from': "projects",
'localField': "project",
'foreignField': "_id",
'as': "project",
}
},
{
'$unwind':
{
'path': '$project',
}
},
{
'$project':
{
'project.is_private': 1,
}
},
{'$match': {'project.is_private': False}},
{'$count': 'tot'}
]
c = current_app.db()['nodes']
# If we provide a query, we extend the first $match step in the aggregation pipeline with
# with the extra parameters (for example node_type)
if query:
pipeline[0]['$match'].update(query)
# Return either a list with one item or an empty list
r = list(c.aggregate(pipeline=pipeline))
count = 0 if not r else r[0]['tot']
return count
def count_users(query=None) -> int:
u = current_app.db()['users']
return u.count(query)
def count_blender_sync(query=None) -> int:
pipeline = [
# 0 Find all startups.blend that are not deleted
{
'$match': {
'_deleted': {'$ne': 'true'},
'name': 'startup.blend',
}
},
# 1 Group them per project (drops any duplicates)
{'$group': {'_id': '$project'}},
# 2 Join the project info
{
'$lookup':
{
'from': "projects",
'localField': "_id",
'foreignField': "_id",
'as': "project",
}
},
# 3 Unwind the project list (there is always only one project)
{
'$unwind':
{
'path': '$project',
}
},
# 4 Find all home projects
{'$match': {'project.category': 'home'}},
{'$count': 'tot'}
]
c = current_app.db()['nodes']
# If we provide a query, we extend the first $match step in the aggregation pipeline with
# with the extra parameters (for example _created)
if query:
pipeline[0]['$match'].update(query)
# Return either a list with one item or an empty list
r = list(c.aggregate(pipeline=pipeline))
count = 0 if not r else r[0]['tot']
return count

View File

@@ -1,57 +0,0 @@
import logging
import datetime
import functools
from flask import Blueprint, jsonify
from cloud.stats import count_nodes, count_users, count_blender_sync
blueprint = Blueprint('cloud.stats', __name__, url_prefix='/s')
log = logging.getLogger(__name__)
@functools.lru_cache()
def get_stats(before: datetime.datetime):
query_comments = {'node_type': 'comment'}
query_assets = {'node_type': 'asset'}
date_query = {}
if before:
date_query = {'_created': {'$lt': before}}
query_comments.update(date_query)
query_assets.update(date_query)
stats = {
'comments': count_nodes(query_comments),
'assets': count_nodes(query_assets),
'users_total': count_users(date_query),
'users_blender_sync': count_blender_sync(date_query),
}
return stats
@blueprint.route('/')
@blueprint.route('/before/<int:before>')
def index(before: int=0):
"""
This endpoint is queried on a daily basis by grafista to retrieve cloud usage
stats. For assets and comments we take into considerations only those who belong
to public projects.
These is the data we retrieve
- Comments count
- Assets count (video, images and files)
- Users count (subscribers count goes via store)
- Blender Sync users
"""
# TODO: Implement project-level metrics (and update ad every child update)
if before:
before = datetime.datetime.strptime(str(before), '%Y%m%d')
else:
today = datetime.date.today()
before = datetime.datetime(today.year, today.month, today.day)
return jsonify(get_stats(before))

View File

@@ -1,72 +0,0 @@
"""Blender Store interface."""
import logging
import typing
from pillar import current_app
log = logging.getLogger(__name__)
def fetch_subscription_info(email: str) -> typing.Optional[dict]:
"""Returns the user info dict from the external subscriptions management server.
:returns: the store user info, or None if the user can't be found or there
was an error communicating. A dict like this is returned:
{
"shop_id": 700,
"cloud_access": 1,
"paid_balance": 314.75,
"balance_currency": "EUR",
"start_date": "2014-08-25 17:05:46",
"expiration_date": "2016-08-24 13:38:45",
"subscription_status": "wc-active",
"expiration_date_approximate": true
}
"""
from requests.adapters import HTTPAdapter
import requests.exceptions
external_subscriptions_server = current_app.config['EXTERNAL_SUBSCRIPTIONS_MANAGEMENT_SERVER']
if log.isEnabledFor(logging.DEBUG):
import urllib.parse
log_email = urllib.parse.quote(email)
log.debug('Connecting to store at %s?blenderid=%s',
external_subscriptions_server, log_email)
# Retry a few times when contacting the store.
s = requests.Session()
s.mount(external_subscriptions_server, HTTPAdapter(max_retries=5))
try:
r = s.get(external_subscriptions_server,
params={'blenderid': email},
verify=current_app.config['TLS_CERT_FILE'],
timeout=current_app.config.get('EXTERNAL_SUBSCRIPTIONS_TIMEOUT_SECS', 10))
except requests.exceptions.ConnectionError as ex:
log.error('Error connecting to %s: %s', external_subscriptions_server, ex)
return None
except requests.exceptions.Timeout as ex:
log.error('Timeout communicating with %s: %s', external_subscriptions_server, ex)
return None
except requests.exceptions.RequestException as ex:
log.error('Some error communicating with %s: %s', external_subscriptions_server, ex)
return None
if r.status_code != 200:
log.warning("Error communicating with %s, code=%i, unable to check "
"subscription status of user %s",
external_subscriptions_server, r.status_code, email)
return None
store_user = r.json()
if log.isEnabledFor(logging.DEBUG):
import json
log.debug('Received JSON from store API: %s',
json.dumps(store_user, sort_keys=False, indent=4))
return store_user

View File

@@ -1,3 +0,0 @@
"""Routes for fetching tagged assets."""

View File

@@ -1,16 +0,0 @@
import logging
import datetime
import functools
from flask import Blueprint, jsonify
blueprint = Blueprint('cloud.tagged', __name__, url_prefix='/tagged')
log = logging.getLogger(__name__)
@blueprint.route('/')
def index():
"""Return all tagged assets as JSON, grouped by tag."""

View File

@@ -1,215 +0,0 @@
"""Blender ID webhooks."""
import functools
import hashlib
import hmac
import json
import logging
import typing
from flask import Blueprint, request
import werkzeug.exceptions as wz_exceptions
from pillar import current_app
from pillar.api.blender_cloud import subscription
from pillar.api.utils.authentication import create_new_user_document, make_unique_username
from pillar.auth import UserClass
blueprint = Blueprint('cloud-webhooks', __name__)
log = logging.getLogger(__name__)
WEBHOOK_MAX_BODY_SIZE = 1024 * 10 # 10 kB is large enough for
def webhook_payload(hmac_secret: str) -> dict:
"""Obtains the webhook payload from the request, verifying its HMAC.
:returns the webhook payload as dictionary.
"""
# Check the content type
if request.content_type != 'application/json':
log.info('request from %s to %s had bad content type %s',
request.remote_addr, request, request.content_type)
raise wz_exceptions.BadRequest('Content type not supported')
# Check the length of the body
if request.content_length > WEBHOOK_MAX_BODY_SIZE:
raise wz_exceptions.BadRequest('Request too large')
body = request.get_data()
if len(body) > request.content_length:
raise wz_exceptions.BadRequest('Larger body than Content-Length header')
# Validate the request
mac = hmac.new(hmac_secret.encode(), body, hashlib.sha256)
req_hmac = request.headers.get('X-Webhook-HMAC', '')
our_hmac = mac.hexdigest()
if not hmac.compare_digest(req_hmac, our_hmac):
log.info('request from %s to %s had bad HMAC %r, expected %r',
request.remote_addr, request, req_hmac, our_hmac)
raise wz_exceptions.BadRequest('Bad HMAC')
try:
return json.loads(body)
except json.JSONDecodeError as ex:
log.warning('request from %s to %s had bad JSON: %s',
request.remote_addr, request, ex)
raise wz_exceptions.BadRequest('Bad JSON')
def score(wh_payload: dict, user: dict) -> int:
"""Determine how likely it is that this is the correct user to modify.
:param wh_payload: the info we received from Blender ID;
see user_modified()
:param user: the user in our database
:return: the score for this user
"""
bid_str = str(wh_payload['id'])
try:
match_on_bid = any((auth['provider'] == 'blender-id' and auth['user_id'] == bid_str)
for auth in user['auth'])
except KeyError:
match_on_bid = False
match_on_old_email = user.get('email', 'none') == wh_payload.get('old_email', 'nothere')
match_on_new_email = user.get('email', 'none') == wh_payload.get('email', 'nothere')
return match_on_bid * 10 + match_on_old_email + match_on_new_email * 2
def insert_or_fetch_user(wh_payload: dict) -> typing.Optional[dict]:
"""Fetch the user from the DB or create it.
Only creates it if the webhook payload indicates they could actually use
Blender Cloud (i.e. demo or subscriber). This prevents us from creating
Cloud accounts for Blender Network users.
:returns the user document, or None when not created.
"""
users_coll = current_app.db('users')
my_log = log.getChild('insert_or_fetch_user')
bid_str = str(wh_payload['id'])
email = wh_payload['email']
# Find the user by their Blender ID, or any of their email addresses.
# We use one query to find all matching users. This is done as a
# consistency check; if more than one user is returned, we know the
# database is inconsistent with Blender ID and can emit a warning
# about this.
query = {'$or': [
{'auth.provider': 'blender-id', 'auth.user_id': bid_str},
{'email': {'$in': [wh_payload['old_email'], email]}},
]}
db_users = users_coll.find(query)
user_count = db_users.count()
if user_count > 1:
# Now we have to pay the price for finding users in one query; we
# have to prioritise them and return the one we think is most reliable.
calc_score = functools.partial(score, wh_payload)
best_score = max(db_users, key=calc_score)
my_log.error('%d users found for query %s, picking user %s (%s)',
user_count, query, best_score['_id'], best_score['email'])
return best_score
if user_count:
db_user = db_users[0]
my_log.debug('found user %s', db_user['email'])
return db_user
# Pretend to create the user, so that we can inspect the resulting
# capabilities. This is more future-proof than looking at the list
# of roles in the webhook payload.
username = make_unique_username(email)
user_doc = create_new_user_document(email, bid_str, username,
provider='blender-id',
full_name=wh_payload['full_name'])
# Figure out the user's eventual roles. These aren't stored in the document yet,
# because that's handled by the badger service.
eventual_roles = [subscription.ROLES_BID_TO_PILLAR[r]
for r in wh_payload.get('roles', [])
if r in subscription.ROLES_BID_TO_PILLAR]
user_ob = UserClass.construct('', user_doc)
user_ob.roles = eventual_roles
user_ob.collect_capabilities()
create = (user_ob.has_cap('subscriber') or
user_ob.has_cap('can-renew-subscription') or
current_app.org_manager.user_is_unknown_member(email))
if not create:
my_log.info('Received update for unknown user %r without Cloud access (caps=%s)',
wh_payload['old_email'], user_ob.capabilities)
return None
# Actually create the user in the database.
r, _, _, status = current_app.post_internal('users', user_doc)
if status != 201:
my_log.error('unable to create user %s: : %r %r', email, status, r)
raise wz_exceptions.InternalServerError('unable to create user')
user_doc.update(r)
my_log.info('created user %r = %s to allow immediate Cloud access', email, user_doc['_id'])
return user_doc
@blueprint.route('/user-modified', methods=['POST'])
def user_modified():
"""Update the local user based on the info from Blender ID.
If the payload indicates the user has access to Blender Cloud (or at least
a renewable subscription), create the user if not already in our DB.
The payload we expect is a dictionary like:
{'id': 12345, # the user's ID in Blender ID
'old_email': 'old@example.com',
'full_name': 'Harry',
'email': 'new@example'com,
'roles': ['role1', 'role2', …]}
"""
my_log = log.getChild('user_modified')
my_log.debug('Received request from %s', request.remote_addr)
hmac_secret = current_app.config['BLENDER_ID_WEBHOOK_USER_CHANGED_SECRET']
payload = webhook_payload(hmac_secret)
my_log.info('payload: %s', payload)
# Update the user
db_user = insert_or_fetch_user(payload)
if not db_user:
my_log.info('Received update for unknown user %r', payload['old_email'])
return '', 204
# Use direct database updates to change the email and full name.
# Also updates the db_user dict so that local_user below will have
# the updated information.
updates = {}
if db_user['email'] != payload['email']:
my_log.info('User changed email from %s to %s', payload['old_email'], payload['email'])
updates['email'] = payload['email']
db_user['email'] = payload['email']
if db_user['full_name'] != payload['full_name']:
my_log.info('User changed full name from %r to %r',
db_user['full_name'], payload['full_name'])
if payload['full_name']:
updates['full_name'] = payload['full_name']
else:
# Fall back to the username when the full name was erased.
updates['full_name'] = db_user['username']
db_user['full_name'] = updates['full_name']
if updates:
users_coll = current_app.db('users')
update_res = users_coll.update_one({'_id': db_user['_id']},
{'$set': updates})
if update_res.matched_count != 1:
my_log.error('Unable to find user %s to update, even though '
'we found them by email address %s',
db_user['_id'], payload['old_email'])
# Defer to Pillar to do the role updates.
local_user = UserClass.construct('', db_user)
subscription.do_update_subscription(local_user, payload)
return '', 204

View File

@@ -1,296 +0,0 @@
#!/usr/bin/env python3
from __future__ import print_function
"""CLI command for sharing an image via Blender Cloud.
Assumes that you are logged in on Blender ID with the Blender ID Add-on.
The user_config_dir and user_data_dir functions come from
https://github.com/ActiveState/appdirs/blob/master/appdirs.py and
are licensed under the MIT license.
"""
import argparse
import json
import mimetypes
import os.path
import pprint
import sys
import webbrowser
from urllib.parse import urljoin
import requests
cli = argparse.Namespace() # CLI args from argparser
sess = requests.Session()
IMAGE_SHARING_GROUP_NODE_NAME = 'Image sharing'
if sys.platform.startswith('java'):
import platform
os_name = platform.java_ver()[3][0]
if os_name.startswith('Windows'): # "Windows XP", "Windows 7", etc.
system = 'win32'
elif os_name.startswith('Mac'): # "Mac OS X", etc.
system = 'darwin'
else: # "Linux", "SunOS", "FreeBSD", etc.
# Setting this to "linux2" is not ideal, but only Windows or Mac
# are actually checked for and the rest of the module expects
# *sys.platform* style strings.
system = 'linux2'
else:
system = sys.platform
def request(method: str, rel_url: str, **kwargs) -> requests.Response:
kwargs.setdefault('auth', (cli.token, ''))
url = urljoin(cli.server_url, rel_url)
return sess.request(method, url, **kwargs)
def get(rel_url: str, **kwargs) -> requests.Response:
return request('GET', rel_url, **kwargs)
def post(rel_url: str, **kwargs) -> requests.Response:
return request('POST', rel_url, **kwargs)
def find_user_id() -> str:
"""Returns the current user ID."""
print(15 * '=', 'User info', 15 * '=')
resp = get('/api/users/me')
resp.raise_for_status()
user_info = resp.json()
print('You are logged in as %(full_name)s (%(_id)s)' % user_info)
return user_info['_id']
def find_home_project_id() -> dict:
resp = get('/api/bcloud/home-project')
resp.raise_for_status()
proj = resp.json()
proj_id = proj['_id']
print('Your home project ID is %s' % proj_id)
return proj_id
def find_image_sharing_group_id(home_project_id, user_id) -> str:
"""Find the top-level image sharing group node."""
node_doc = {
'project': home_project_id,
'node_type': 'group',
'name': IMAGE_SHARING_GROUP_NODE_NAME,
'user': user_id,
}
resp = get('/api/nodes', params={'where': json.dumps(node_doc)})
resp.raise_for_status()
items = resp.json()['_items']
if not items:
print('Share group not found, creating one.')
node_doc.update({
'properties': {},
})
resp = post('/api/nodes', json=node_doc)
resp.raise_for_status()
share_group = resp.json()
else:
share_group = items[0]
# print('Share group:', share_group)
return share_group['_id']
def upload_image():
user_id = find_user_id()
home_project_id = find_home_project_id()
group_id = find_image_sharing_group_id(home_project_id, user_id)
basename = os.path.basename(cli.imgfile)
print('Sharing group ID is %s' % group_id)
# Upload the image to the project.
print('Uploading %r' % cli.imgfile)
mimetype, _ = mimetypes.guess_type(cli.imgfile, strict=False)
with open(cli.imgfile, mode='rb') as infile:
resp = post('api/storage/stream/%s' % home_project_id,
files={'file': (basename, infile, mimetype)})
resp.raise_for_status()
file_upload_resp = resp.json()
file_upload_status = file_upload_resp.get('_status') or file_upload_resp.get('status')
if file_upload_status != 'ok':
raise ValueError('Received bad status %s from Pillar: %s' %
(file_upload_status, json.dumps(file_upload_resp)))
file_id = file_upload_resp['file_id']
print('File ID is', file_id)
# Create the asset node
asset_node = {
'project': home_project_id,
'node_type': 'asset',
'name': basename,
'parent': group_id,
'properties': {
'content_type': mimetype,
'file': file_id,
},
}
resp = post('api/nodes', json=asset_node)
resp.raise_for_status()
node_info = resp.json()
node_id = node_info['_id']
print('Created asset node', node_id)
# Share the node to get a public URL.
resp = post('api/nodes/%s/share' % node_id)
resp.raise_for_status()
share_info = resp.json()
print(json.dumps(share_info, indent=4))
url = share_info.get('short_link')
print('Opening %s in a browser' % url)
webbrowser.open_new_tab(url)
def find_credentials():
"""Finds BlenderID credentials.
:rtype: str
:returns: the authentication token to use.
"""
import glob
# Find BlenderID profile file.
configpath = user_config_dir('blender', 'Blender Foundation', roaming=True)
found = glob.glob(os.path.join(configpath, '*'))
for confpath in reversed(sorted(found)):
profiles_path = os.path.join(confpath, 'config', 'blender_id', 'profiles.json')
if not os.path.exists(profiles_path):
continue
print('Reading credentials from %s' % profiles_path)
with open(profiles_path) as infile:
profiles = json.load(infile, encoding='utf8')
if profiles:
break
else:
print('Unable to find Blender ID credentials. Log in with the Blender ID add-on in '
'Blender first.')
raise SystemExit()
active_profile = profiles[u'active_profile']
profile = profiles[u'profiles'][active_profile]
print('Logging in as %s' % profile[u'username'])
return profile[u'token']
def main():
global cli
parser = argparse.ArgumentParser()
parser.add_argument('imgfile', help='The image file to share.')
parser.add_argument('-u', '--server-url', default='https://cloud.blender.org/',
help='URL of the Flamenco server.')
parser.add_argument('-t', '--token',
help='Authentication token to use. If not given, your token from the '
'Blender ID add-on is used.')
cli = parser.parse_args()
if not cli.token:
cli.token = find_credentials()
upload_image()
def user_config_dir(appname=None, appauthor=None, version=None, roaming=False):
r"""Return full path to the user-specific config dir for this application.
"appname" is the name of application.
If None, just the system directory is returned.
"appauthor" (only used on Windows) is the name of the
appauthor or distributing body for this application. Typically
it is the owning company name. This falls back to appname. You may
pass False to disable it.
"version" is an optional version path element to append to the
path. You might want to use this if you want multiple versions
of your app to be able to run independently. If used, this
would typically be "<major>.<minor>".
Only applied when appname is present.
"roaming" (boolean, default False) can be set True to use the Windows
roaming appdata directory. That means that for users on a Windows
network setup for roaming profiles, this user data will be
sync'd on login. See
<http://technet.microsoft.com/en-us/library/cc766489(WS.10).aspx>
for a discussion of issues.
Typical user config directories are:
Mac OS X: same as user_data_dir
Unix: ~/.config/<AppName> # or in $XDG_CONFIG_HOME, if defined
Win *: same as user_data_dir
For Unix, we follow the XDG spec and support $XDG_CONFIG_HOME.
That means, by default "~/.config/<AppName>".
"""
if system in {"win32", "darwin"}:
path = user_data_dir(appname, appauthor, None, roaming)
else:
path = os.getenv('XDG_CONFIG_HOME', os.path.expanduser("~/.config"))
if appname:
path = os.path.join(path, appname)
if appname and version:
path = os.path.join(path, version)
return path
def user_data_dir(appname=None, appauthor=None, version=None, roaming=False):
r"""Return full path to the user-specific data dir for this application.
"appname" is the name of application.
If None, just the system directory is returned.
"appauthor" (only used on Windows) is the name of the
appauthor or distributing body for this application. Typically
it is the owning company name. This falls back to appname. You may
pass False to disable it.
"version" is an optional version path element to append to the
path. You might want to use this if you want multiple versions
of your app to be able to run independently. If used, this
would typically be "<major>.<minor>".
Only applied when appname is present.
"roaming" (boolean, default False) can be set True to use the Windows
roaming appdata directory. That means that for users on a Windows
network setup for roaming profiles, this user data will be
sync'd on login. See
<http://technet.microsoft.com/en-us/library/cc766489(WS.10).aspx>
for a discussion of issues.
Typical user data directories are:
Mac OS X: ~/Library/Application Support/<AppName>
Unix: ~/.local/share/<AppName> # or in $XDG_DATA_HOME, if defined
Win XP (not roaming): C:\Documents and Settings\<username>\Application Data\<AppAuthor>\<AppName>
Win XP (roaming): C:\Documents and Settings\<username>\Local Settings\Application Data\<AppAuthor>\<AppName>
Win 7 (not roaming): C:\Users\<username>\AppData\Local\<AppAuthor>\<AppName>
Win 7 (roaming): C:\Users\<username>\AppData\Roaming\<AppAuthor>\<AppName>
For Unix, we follow the XDG spec and support $XDG_DATA_HOME.
That means, by default "~/.local/share/<AppName>".
"""
if system == "win32":
raise RuntimeError("Sorry, Windows is not supported for now.")
elif system == 'darwin':
path = os.path.expanduser('~/Library/Application Support/')
if appname:
path = os.path.join(path, appname)
else:
path = os.getenv('XDG_DATA_HOME', os.path.expanduser("~/.local/share"))
if appname:
path = os.path.join(path, appname)
if appname and version:
path = os.path.join(path, version)
return path
if __name__ == '__main__':
main()

View File

@@ -1,32 +0,0 @@
import os
DEBUG = True
BLENDER_ID_ENDPOINT = 'http://id.local:8000/'
SERVER_NAME = 'cloud.local:5001'
SCHEME = 'http'
PILLAR_SERVER_ENDPOINT = f'{SCHEME}://{SERVER_NAME}/api/'
os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = 'true'
os.environ['PILLAR_MONGO_DBNAME'] = 'cloud'
os.environ['PILLAR_MONGO_PORT'] = '27017'
os.environ['PILLAR_MONGO_HOST'] = 'mongo'
os.environ['PILLAR_SERVER_ENDPOINT'] = PILLAR_SERVER_ENDPOINT
SECRET_KEY = '##DEFINE##'
OAUTH_CREDENTIALS = {
'blender-id': {
'id': 'CLOUD-OF-SNOWFLAKES-42',
'secret': '##DEFINE##',
}
}
MAIN_PROJECT_ID = '##DEFINE##'
URLER_SERVICE_AUTH_TOKEN = '##DEFINE##'
ZENCODER_API_KEY = '##DEFINE##'
ZENCODER_NOTIFICATIONS_SECRET = '##DEFINE##'
ZENCODER_NOTIFICATIONS_URL = 'http://zencoderfetcher/'

124
deploy.sh Executable file
View File

@@ -0,0 +1,124 @@
#!/bin/bash -e
# Deploys the current production branch to the production machine.
PROJECT_NAME="blender-cloud"
DOCKER_NAME="blender_cloud"
REMOTE_ROOT="/data/git/${PROJECT_NAME}"
SSH="ssh -o ClearAllForwardings=yes cloud.blender.org"
# macOS does not support readlink -f, so we use greadlink instead
if [[ `uname` == 'Darwin' ]]; then
command -v greadlink 2>/dev/null 2>&1 || { echo >&2 "Install greadlink using brew."; exit 1; }
readlink='greadlink'
else
readlink='readlink'
fi
ROOT="$(dirname "$($readlink -f "$0")")"
cd ${ROOT}
# Check that we're on production branch.
if [ $(git rev-parse --abbrev-ref HEAD) != "production" ]; then
echo "You are NOT on the production branch, refusing to deploy." >&2
exit 1
fi
# Check that production branch has been pushed.
if [ -n "$(git log origin/production..production --oneline)" ]; then
echo "WARNING: not all changes to the production branch have been pushed."
echo "Press [ENTER] to continue deploying current origin/production, CTRL+C to abort."
read dummy
fi
function find_module()
{
MODULE_NAME=$1
MODULE_DIR=$(python <<EOT
from __future__ import print_function
import os.path
try:
import ${MODULE_NAME}
except ImportError:
raise SystemExit('${MODULE_NAME} not found on Python path. Are you in the correct venv?')
print(os.path.dirname(os.path.dirname(${MODULE_NAME}.__file__)))
EOT
)
if [ $(git -C $MODULE_DIR rev-parse --abbrev-ref HEAD) != "production" ]; then
echo "${MODULE_NAME}: ($MODULE_DIR) NOT on the production branch, refusing to deploy." >&2
exit 1
fi
echo $MODULE_DIR
}
# Find our modules
PILLAR_DIR=$(find_module pillar)
ATTRACT_DIR=$(find_module attract)
FLAMENCO_DIR=$(find_module flamenco)
echo "Pillar : $PILLAR_DIR"
echo "Attract : $ATTRACT_DIR"
echo "Flamenco: $FLAMENCO_DIR"
if [ -z "$PILLAR_DIR" -o -z "$ATTRACT_DIR" -o -z "$FLAMENCO_DIR" ];
then
exit 1
fi
# SSH to cloud to pull all files in
function git_pull() {
PROJECT_NAME="$1"
BRANCH="$2"
REMOTE_ROOT="/data/git/${PROJECT_NAME}"
echo "==================================================================="
echo "UPDATING FILES ON ${PROJECT_NAME}"
${SSH} git -C ${REMOTE_ROOT} fetch origin ${BRANCH}
${SSH} git -C ${REMOTE_ROOT} log origin/${BRANCH}..${BRANCH} --oneline
${SSH} git -C ${REMOTE_ROOT} merge --ff-only origin/${BRANCH}
}
git_pull pillar-python-sdk master
git_pull pillar production
git_pull attract production
git_pull flamenco production
git_pull blender-cloud production
# Update the virtualenv
#${SSH} -t docker exec ${DOCKER_NAME} /data/venv/bin/pip install -U -r ${REMOTE_ROOT}/requirements.txt --exists-action w
# RSync the world
$ATTRACT_DIR/rsync_ui.sh
$FLAMENCO_DIR/rsync_ui.sh
./rsync_ui.sh
# Notify Bugsnag of this new deploy.
echo
echo "==================================================================="
GIT_REVISION=$(${SSH} git -C ${REMOTE_ROOT} describe --always)
echo "Notifying Bugsnag of this new deploy of revision ${GIT_REVISION}."
BUGSNAG_API_KEY=$(${SSH} python -c "\"import sys; sys.path.append('${REMOTE_ROOT}'); import config_local; print(config_local.BUGSNAG_API_KEY)\"")
curl --data "apiKey=${BUGSNAG_API_KEY}&revision=${GIT_REVISION}" https://notify.bugsnag.com/deploy
echo
# Wait for [ENTER] to restart the server
echo
echo "==================================================================="
echo "NOTE: If you want to edit config_local.py on the server, do so now."
echo "NOTE: Press [ENTER] to continue and restart the server process."
read dummy
${SSH} docker exec ${DOCKER_NAME} apache2ctl graceful
echo "Server process restarted"
echo
echo "==================================================================="
echo "Clearing front page from Redis cache."
${SSH} docker exec redis redis-cli DEL pwview//
echo
echo "==================================================================="
echo "Deploy of ${PROJECT_NAME} is done."
echo "==================================================================="

View File

@@ -1,116 +0,0 @@
#!/bin/bash -e
DEPLOY_BRANCH=${DEPLOY_BRANCH:-production}
# macOS does not support readlink -f, so we use greadlink instead
if [[ `uname` == 'Darwin' ]]; then
command -v greadlink 2>/dev/null 2>&1 || { echo >&2 "Install greadlink using brew."; exit 1; }
readlink='greadlink'
else
readlink='readlink'
fi
ROOT="$(dirname "$(dirname "$($readlink -f "$0")")")"
DEPLOYDIR="$ROOT/docker/4_run/deploy"
PROJECT_NAME="$(basename $ROOT)"
if [ -e $DEPLOYDIR ]; then
echo "$DEPLOYDIR already exists, press [ENTER] to DESTROY AND DEPLOY, Ctrl+C to abort."
read dummy
rm -rf $DEPLOYDIR
else
echo -n "Deploying to $DEPLOYDIR"
echo "press [ENTER] to continue, Ctrl+C to abort."
read dummy
fi
cd ${ROOT}
mkdir -p $DEPLOYDIR
REMOTE_ROOT="$DEPLOYDIR/$PROJECT_NAME"
if [ -z "$SKIP_BRANCH_CHECK" ]; then
# Check that we're on production branch.
if [ $(git rev-parse --abbrev-ref HEAD) != "$DEPLOY_BRANCH" ]; then
echo "You are NOT on the $DEPLOY_BRANCH branch, refusing to deploy." >&2
exit 1
fi
# Check that production branch has been pushed.
if [ -n "$(git log origin/$DEPLOY_BRANCH..$DEPLOY_BRANCH --oneline)" ]; then
echo "WARNING: not all changes to the $DEPLOY_BRANCH branch have been pushed."
echo "Press [ENTER] to continue deploying current origin/$DEPLOY_BRANCH, CTRL+C to abort."
read dummy
fi
fi
function find_module()
{
MODULE_NAME=$1
MODULE_DIR=$(python <<EOT
from __future__ import print_function
import os.path
try:
import ${MODULE_NAME}
except ImportError:
raise SystemExit('${MODULE_NAME} not found on Python path. Are you in the correct venv?')
print(os.path.dirname(os.path.dirname(${MODULE_NAME}.__file__)))
EOT
)
echo $MODULE_DIR
}
# Find our modules
echo "==================================================================="
echo "LOCAL MODULE LOCATIONS"
PILLAR_DIR=$(find_module pillar)
ATTRACT_DIR=$(find_module attract)
FLAMENCO_DIR=$(find_module flamenco)
SVNMAN_DIR=$(find_module svnman)
SDK_DIR=$(find_module pillarsdk)
echo "Pillar : $PILLAR_DIR"
echo "Attract : $ATTRACT_DIR"
echo "Flamenco: $FLAMENCO_DIR"
echo "SVNMan : $SVNMAN_DIR"
echo "SDK : $SDK_DIR"
if [ -z "$PILLAR_DIR" -o -z "$ATTRACT_DIR" -o -z "$FLAMENCO_DIR" -o -z "$SVNMAN_DIR" -o -z "$SDK_DIR" ];
then
exit 1
fi
function git_clone() {
PROJECT_NAME="$1"
BRANCH="$2"
LOCAL_ROOT="$3"
echo "==================================================================="
echo "CLONING REPO ON $PROJECT_NAME @$BRANCH"
URL=$(git -C $LOCAL_ROOT remote get-url origin)
git -C $DEPLOYDIR clone --depth 1 --branch $BRANCH $URL $PROJECT_NAME
}
git_clone pillar-python-sdk master $SDK_DIR
git_clone pillar $DEPLOY_BRANCH $PILLAR_DIR
git_clone attract $DEPLOY_BRANCH $ATTRACT_DIR
git_clone flamenco $DEPLOY_BRANCH $FLAMENCO_DIR
git_clone pillar-svnman $DEPLOY_BRANCH $SVNMAN_DIR
git_clone blender-cloud $DEPLOY_BRANCH $ROOT
# Gulp everywhere
GULP=$ROOT/node_modules/.bin/gulp
if [ ! -e $GULP -o gulpfile.js -nt $GULP ]; then
npm install
touch $GULP # installer doesn't always touch this after a build, so we do.
fi
$GULP --cwd $DEPLOYDIR/pillar --production
$GULP --cwd $DEPLOYDIR/attract --production
$GULP --cwd $DEPLOYDIR/flamenco --production
$GULP --cwd $DEPLOYDIR/pillar-svnman --production
$GULP --cwd $DEPLOYDIR/blender-cloud --production
echo
echo "==================================================================="
echo "Deploy of ${PROJECT_NAME} is ready for dockerisation."
echo "==================================================================="

View File

@@ -1,81 +0,0 @@
#!/bin/bash -e
# macOS does not support readlink -f, so we use greadlink instead
if [[ `uname` == 'Darwin' ]]; then
command -v greadlink 2>/dev/null 2>&1 || { echo >&2 "Install greadlink using brew."; exit 1; }
readlink='greadlink'
else
readlink='readlink'
fi
ROOT="$(dirname "$(dirname "$($readlink -f "$0")")")"
PROJECT_NAME="$(basename $ROOT)"
DOCKER_DEPLOYDIR="$ROOT/docker/4_run/deploy"
DOCKER_IMAGE="armadillica/blender_cloud:latest"
REMOTE_SECRET_CONFIG_DIR="/data/config"
REMOTE_DOCKER_COMPOSE_DIR="/root/docker"
#################################################################################
case $1 in
cloud*)
DEPLOYHOST="$1"
;;
*)
echo "Use $0 cloud{nr}|cloud.blender.org" >&2
exit 1
esac
SSH_OPTS="-o ClearAllForwardings=yes -o PermitLocalCommand=no"
SSH="ssh $SSH_OPTS $DEPLOYHOST"
SCP="scp $SSH_OPTS"
echo -n "Deploying to $DEPLOYHOST"
if ! ping $DEPLOYHOST -q -c 1 -W 2 >/dev/null; then
echo "host $DEPLOYHOST cannot be pinged, refusing to deploy." >&2
exit 2
fi
cat <<EOT
[ping OK]
Make sure that you have pushed the $DOCKER_IMAGE
docker image to Docker Hub.
press [ENTER] to continue, Ctrl+C to abort.
EOT
read dummy
#################################################################################
echo "==================================================================="
echo "Bringing remote Docker up to date…"
$SSH mkdir -p $REMOTE_DOCKER_COMPOSE_DIR
$SCP \
$ROOT/docker/{docker-compose.yml,renew-letsencrypt.sh,mongo-backup.{cron,sh}} \
$DEPLOYHOST:$REMOTE_DOCKER_COMPOSE_DIR
$SSH -T <<EOT
set -e
cd $REMOTE_DOCKER_COMPOSE_DIR
docker pull $DOCKER_IMAGE
docker-compose up -d
echo
echo "==================================================================="
echo "Clearing front page from Redis cache."
docker exec redis redis-cli DEL pwview//
EOT
# Notify Sentry of this new deploy.
# See https://sentry.io/blender-institute/blender-cloud/settings/release-tracking/
# and https://docs.sentry.io/api/releases/post-organization-releases/
# and https://sentry.io/api/
echo
echo "==================================================================="
REVISION=$(date +'%Y%m%d.%H%M%S.%Z')
echo "Notifying Sentry of this new deploy of revision $REVISION."
SENTRY_RELEASE_URL="$($SSH env PYTHONPATH="$REMOTE_SECRET_CONFIG_DIR" python3 -c "\"import config_secrets; print(config_secrets.SENTRY_RELEASE_URL)\"")"
curl -s "$SENTRY_RELEASE_URL" -XPOST -H 'Content-Type: application/json' -d "{\"version\": \"$REVISION\"}" | json_pp
echo
echo
echo "==================================================================="
echo "Deploy to $DEPLOYHOST done."
echo "==================================================================="

View File

@@ -1 +0,0 @@
build-quick.sh

View File

@@ -1,34 +0,0 @@
#!/bin/bash -e
# macOS does not support readlink -f, so we use greadlink instead
if [[ `uname` == 'Darwin' ]]; then
command -v greadlink 2>/dev/null 2>&1 || { echo >&2 "Install greadlink using brew."; exit 1; }
readlink='greadlink'
else
readlink='readlink'
fi
ROOT="$(dirname "$(dirname "$($readlink -f "$0")")")"
DOCKERDIR="$ROOT/docker/4_run"
case "$(basename "$0")" in
build-quick.sh)
pushd "$ROOT/docker/4_run"
./build.sh
;;
build-all.sh)
pushd "$ROOT/docker"
./full_rebuild.sh
;;
*)
echo "Unknown script $0, aborting" >&2
exit 1
esac
popd
echo
echo "Press [ENTER] to push the new Docker image."
read dummy
docker push armadillica/blender_cloud:latest
echo
echo "Build is done, ready to update the server."

View File

@@ -1,10 +0,0 @@
FROM ubuntu:18.04
LABEL maintainer="Sybren A. Stüvel <sybren@blender.studio>"
RUN set -ex; \
apt-get update; \
DEBIAN_FRONTEND=noninteractive apt-get install \
-qyy -o APT::Install-Recommends=false -o APT::Install-Suggests=false \
tzdata openssl ca-certificates locales; \
locale-gen en_US.UTF-8 en_GB.UTF-8 nl_NL.UTF-8
ENV LANG en_US.UTF-8

16
docker/1_base/base.docker Executable file
View File

@@ -0,0 +1,16 @@
FROM ubuntu:16.04
MAINTAINER Francesco Siddi <francesco@blender.org>
RUN apt-get update && apt-get install -qyy \
-o APT::Install-Recommends=false -o APT::Install-Suggests=false \
python-pip libffi6 openssl ffmpeg rsyslog logrotate
RUN mkdir -p /data/git/pillar \
&& mkdir -p /data/storage \
&& mkdir -p /data/config \
&& mkdir -p /data/venv \
&& mkdir -p /data/wheelhouse
RUN pip install virtualenv
RUN virtualenv /data/venv
RUN . /data/venv/bin/activate && pip install -U pip && pip install wheel

3
docker/1_base/build.sh Executable file → Normal file
View File

@@ -1,4 +1,3 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Uses --no-cache to always get the latest upstream (security) upgrades. docker build -t pillar_base -f base.docker .;
exec docker build --no-cache "$@" -t pillar_base .

View File

@@ -0,0 +1,3 @@
#!/usr/bin/env bash
. /data/venv/bin/activate && pip wheel --wheel-dir=/data/wheelhouse -r /requirements.txt

26
docker/2_build/build.docker Executable file
View File

@@ -0,0 +1,26 @@
FROM pillar_base
MAINTAINER Francesco Siddi <francesco@blender.org>
RUN apt-get update && apt-get install -qy \
git \
gcc \
libffi-dev \
libssl-dev \
pypy-dev \
python-dev \
python-imaging \
zlib1g-dev \
libjpeg-dev \
libtiff-dev \
python-crypto \
python-openssl
ENV WHEELHOUSE=/data/wheelhouse
ENV PIP_WHEEL_DIR=/data/wheelhouse
ENV PIP_FIND_LINKS=/data/wheelhouse
VOLUME /data/wheelhouse
ADD requirements.txt /requirements.txt
ADD build-wheels.sh /build-wheels.sh
ENTRYPOINT ["bash", "build-wheels.sh"]

11
docker/2_build/build.sh Executable file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
mkdir -p ../3_run/wheelhouse;
cp ../../requirements.txt .;
docker build -t pillar_build -f build.docker .;
docker run --rm \
-v "$(pwd)"/../3_run/wheelhouse:/data/wheelhouse \
pillar_build;
rm requirements.txt;

View File

@@ -1 +0,0 @@
c3f30a0aff425dda77d19e02f420d6ba Python-3.6.6.tar.xz

View File

@@ -1,61 +0,0 @@
#!/usr/bin/env bash
set -e
# macOS does not support readlink -f, so we use greadlink instead
if [ $(uname) == 'Darwin' ]; then
command -v greadlink 2>/dev/null 2>&1 || { echo >&2 "Install greadlink using brew."; exit 1; }
readlink='greadlink'
else
readlink='readlink'
fi
PYTHONTARGET=$($readlink -f ./python)
mkdir -p "$PYTHONTARGET"
echo "Python will be built to $PYTHONTARGET"
docker build -t pillar_build -f buildpy.docker .
# Use the docker image to build Python 3.6 and mod-wsgi
GID=$(id -g)
docker run --rm -i \
-v "$PYTHONTARGET:/opt/python" \
pillar_build <<EOT
set -e
cd \$PYTHONSOURCE
./configure \
--prefix=/opt/python \
--enable-ipv6 \
--enable-shared \
--with-ensurepip=upgrade
make -j8 install
# Make sure we can run Python
ldconfig
# Upgrade pip
/opt/python/bin/python3 -m pip install -U pip
# Build mod-wsgi-py3 for Python 3.6
cd /dpkg/mod-wsgi-*
./configure --with-python=/opt/python/bin/python3
make -j8 install
mkdir -p /opt/python/mod-wsgi
cp /usr/lib/apache2/modules/mod_wsgi.so /opt/python/mod-wsgi
chown -R $UID:$GID /opt/python/*
EOT
# Strip some stuff we don't need from the Python install.
rm -rf $PYTHONTARGET/lib/python3.*/test
rm -rf $PYTHONTARGET/lib/python3.*/config-3.*/libpython3.*.a
find $PYTHONTARGET/lib -name '*.so.*' -o -name '*.so' | while read libname; do
chmod u+w "$libname"
strip "$libname"
done
# Create another docker image which contains the actual Python.
# This one will serve as base for the Wheel builder and the
# production image.
docker build -t armadillica/pillar_py:3.6 -f includepy.docker .

View File

@@ -1,35 +0,0 @@
FROM pillar_base
LABEL maintainer="Sybren A. Stüvel <sybren@blender.studio>"
RUN sed -i 's/^# deb-src/deb-src/' /etc/apt/sources.list && \
apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -qy \
build-essential \
apache2-dev \
checkinstall \
curl
RUN apt-get build-dep -y python3.6
ADD Python-3.6.6.tar.xz.md5 /Python-3.6.6.tar.xz.md5
# Install Python sources
RUN curl -O https://www.python.org/ftp/python/3.6.6/Python-3.6.6.tar.xz && \
md5sum -c Python-3.6.6.tar.xz.md5 && \
tar xf Python-3.6.6.tar.xz && \
rm -v Python-3.6.6.tar.xz
# Install mod-wsgi sources
RUN mkdir -p /dpkg && cd /dpkg && apt-get source libapache2-mod-wsgi-py3
# To be able to install Python outside the docker.
VOLUME /opt/python
# To be able to run Python; after building, ldconfig has to be re-run to do this.
# This makes it easier to use Python right after building (for example to build
# mod-wsgi for Python 3.6).
RUN echo /opt/python/lib > /etc/ld.so.conf.d/python.conf
RUN ldconfig
ENV PATH=/opt/python/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ENV PYTHONSOURCE=/Python-3.6.6

View File

@@ -1,13 +0,0 @@
FROM pillar_base
LABEL maintainer="Sybren A. Stüvel <sybren@blender.studio>"
ADD python /opt/python
RUN echo /opt/python/lib > /etc/ld.so.conf.d/python.conf
RUN ldconfig
RUN echo Python is installed in /opt/python/ > README.python
ENV PATH=/opt/python/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
RUN cd /opt/python/bin && \
ln -s python3 python

View File

@@ -1,20 +0,0 @@
FROM armadillica/pillar_py:3.6
LABEL maintainer="Sybren A. Stüvel <sybren@blender.studio>"
RUN set -ex; \
apt-get update; \
DEBIAN_FRONTEND=noninteractive apt-get install -qy \
git \
build-essential \
checkinstall \
libffi-dev \
libssl-dev \
libjpeg-dev \
zlib1g-dev
ENV WHEELHOUSE=/data/wheelhouse
ENV PIP_WHEEL_DIR=/data/wheelhouse
ENV PIP_FIND_LINKS=/data/wheelhouse
RUN mkdir -p $WHEELHOUSE
VOLUME /data/wheelhouse

View File

@@ -1,45 +0,0 @@
#!/usr/bin/env bash
set -e
# macOS does not support readlink -f, so we use greadlink instead
if [ $(uname) == 'Darwin' ]; then
command -v greadlink 2>/dev/null 2>&1 || { echo >&2 "Install greadlink using brew."; exit 1; }
readlink='greadlink'
else
readlink='readlink'
fi
TOPDEVDIR="$($readlink -f ../../..)"
echo "Top-level development dir is $TOPDEVDIR"
WHEELHOUSE="$($readlink -f ../4_run/wheelhouse)"
if [ -z "$WHEELHOUSE" ]; then
echo "Error, ../4_run might not exist." >&2
exit 2
fi
echo "Wheelhouse is $WHEELHOUSE"
mkdir -p "$WHEELHOUSE"
docker build -t pillar_wheelbuilder .
GID=$(id -g)
docker run --rm -i \
-v "$WHEELHOUSE:/data/wheelhouse" \
-v "$TOPDEVDIR:/data/topdev" \
pillar_wheelbuilder <<EOT
set -e
# Build wheels for all dependencies.
cd /data/topdev/blender-cloud
pip3 install wheel
pip3 wheel --wheel-dir=/data/wheelhouse -r requirements.txt
chown -R $UID:$GID /data/wheelhouse
# Install the dependencies so that we can get a full freeze.
pip3 install --no-index --find-links=/data/wheelhouse -r requirements.txt
pip3 freeze | grep -v '^-[ef] ' > /data/wheelhouse/requirements.txt
EOT
# Remove our own projects, they shouldn't be installed as wheel (for now).
rm -f $WHEELHOUSE/{attract,flamenco,pillar,pillarsdk}*.whl

View File

@@ -0,0 +1,39 @@
<VirtualHost *:80>
# EnableSendfile on
XSendFile on
XSendFilePath /data/storage/pillar
XSendFilePath /data/git/pillar
XSendFilePath /data/venv/lib/python2.7/site-packages/attract/static/
XSendFilePath /data/venv/lib/python2.7/site-packages/flamenco/static/
XsendFilePath /data/git/blender-cloud
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
# Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the loglevel for particular
# modules, e.g.
# LogLevel info ssl:warn
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
WSGIDaemonProcess cloud processes=4 threads=1 maximum-requests=10000
WSGIPassAuthorization On
WSGIScriptAlias / /data/git/blender-cloud/runserver.wsgi \
process-group=cloud application-group=%{GLOBAL}
<Directory /data/git/blender-cloud>
<Files runserver.wsgi>
Require all granted
</Files>
</Directory>
# Temporary edit to remap the old cloudapi.blender.org to cloud.blender.org/api
RewriteEngine On
RewriteCond "%{HTTP_HOST}" "^cloudapi\.blender\.org" [NC]
RewriteRule (.*) /api$1 [PT]
</VirtualHost>

5
docker/3_run/build.sh Executable file
View File

@@ -0,0 +1,5 @@
#!/usr/bin/env bash
cp ../../requirements.txt .;
docker build -t armadillica/blender_cloud -f run.docker .;
rm requirements.txt;

View File

@@ -0,0 +1,25 @@
#!/usr/bin/env bash
if [ ! -f /installed ]; then
echo "Installing pillar and pillar-sdk"
# TODO: curretly doing pip install -e takes a long time, so we symlink
# . /data/venv/bin/activate && pip install -e /data/git/pillar
ln -s /data/git/pillar/pillar /data/venv/lib/python2.7/site-packages/pillar
# . /data/venv/bin/activate && pip install -e /data/git/attract
ln -s /data/git/attract/attract /data/venv/lib/python2.7/site-packages/attract
# . /data/venv/bin/activate && pip install -e /data/git/flamenco/packages/flamenco
ln -s /data/git/flamenco/packages/flamenco/flamenco/ /data/venv/lib/python2.7/site-packages/flamenco
# . /data/venv/bin/activate && pip install -e /data/git/pillar-python-sdk
ln -s /data/git/pillar-python-sdk/pillarsdk /data/venv/lib/python2.7/site-packages/pillarsdk
touch installed
fi
if [ "$DEV" = "true" ]; then
echo "Running in development mode"
cd /data/git/blender-cloud
bash /manage.sh runserver --host='0.0.0.0'
else
# Run Apache
a2enmod rewrite
/usr/sbin/apache2ctl -D FOREGROUND
fi

5
docker/3_run/manage.sh Executable file
View File

@@ -0,0 +1,5 @@
#!/usr/bin/env bash -e
. /data/venv/bin/activate
cd /data/git/blender-cloud
python manage.py "$@"

46
docker/3_run/run.docker Executable file
View File

@@ -0,0 +1,46 @@
FROM pillar_base
RUN apt-get update && apt-get install -qyy \
-o APT::Install-Recommends=true -o APT::Install-Suggests=false \
git \
apache2 \
libapache2-mod-wsgi \
libapache2-mod-xsendfile \
libjpeg8 \
libtiff5 \
nano vim curl \
&& rm -rf /var/lib/apt/lists/*
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_PID_FILE /var/run/apache2.pid
ENV APACHE_RUN_DIR /var/run/apache2
ENV APACHE_LOCK_DIR /var/lock/apache2
RUN mkdir -p $APACHE_RUN_DIR $APACHE_LOCK_DIR $APACHE_LOG_DIR
ADD requirements.txt /requirements.txt
ADD wheelhouse /data/wheelhouse
RUN . /data/venv/bin/activate \
&& pip install --no-index --find-links=/data/wheelhouse -r requirements.txt \
&& rm /requirements.txt
VOLUME /data/git/blender-cloud
VOLUME /data/git/pillar
VOLUME /data/git/pillar-python-sdk
VOLUME /data/config
VOLUME /data/storage
ENV USE_X_SENDFILE True
EXPOSE 80
EXPOSE 5000
ADD apache2.conf /etc/apache2/apache2.conf
ADD 000-default.conf /etc/apache2/sites-available/000-default.conf
ADD docker-entrypoint.sh /docker-entrypoint.sh
ADD manage.sh /manage.sh
ENTRYPOINT ["bash", "/docker-entrypoint.sh"]

View File

@@ -1,64 +0,0 @@
FROM armadillica/pillar_py:3.6
LABEL maintainer="Sybren A. Stüvel <sybren@blender.studio>"
RUN set -ex; \
apt-get update; \
DEBIAN_FRONTEND=noninteractive apt-get install -qy \
-o APT::Install-Recommends=false -o APT::Install-Suggests=false \
git \
apache2 \
libapache2-mod-xsendfile \
libjpeg8 \
libtiff5 \
ffmpeg \
rsyslog logrotate \
nano vim-tiny curl; \
rm -rf /var/lib/apt/lists/*
RUN ln -s /usr/bin/vim.tiny /usr/bin/vim
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_PID_FILE /var/run/apache2.pid
ENV APACHE_RUN_DIR /var/run/apache2
ENV APACHE_LOCK_DIR /var/lock/apache2
RUN mkdir -p $APACHE_RUN_DIR $APACHE_LOCK_DIR $APACHE_LOG_DIR
ADD wheelhouse /data/wheelhouse
RUN pip3 install --no-index --find-links=/data/wheelhouse -r /data/wheelhouse/requirements.txt
VOLUME /data/config
VOLUME /data/storage
VOLUME /var/log
ENV USE_X_SENDFILE True
EXPOSE 80
EXPOSE 5000
ADD apache/wsgi-py36.* /etc/apache2/mods-available/
RUN a2enmod rewrite && a2enmod wsgi-py36
ADD apache/apache2.conf /etc/apache2/apache2.conf
ADD apache/000-default.conf /etc/apache2/sites-available/000-default.conf
ADD apache/logrotate.conf /etc/logrotate.d/apache2
ADD *.sh /
# Remove some empty top-level directories we won't use anyway.
RUN rmdir /media /home 2>/dev/null || true
# This file includes some useful commands to have in the shell history
# for easy access.
ADD bash_history /root/.bash_history
ENTRYPOINT /docker-entrypoint.sh
# Add the most-changing files as last step for faster rebuilds.
ADD config_local.py /data/git/blender-cloud/
ADD deploy /data/git
RUN python3 -c "import re, secrets; \
f = open('/data/git/blender-cloud/config_local.py', 'a'); \
h = re.sub(r'[_.~-]', '', secrets.token_urlsafe())[:8]; \
print(f'STATIC_FILE_HASH = {h!r}', file=f)"

View File

@@ -1,54 +0,0 @@
<VirtualHost *:80>
XSendFile on
XSendFilePath /data/storage/pillar
XSendFilePath /data/git/pillar/pillar/web/static/
XSendFilePath /data/git/attract/attract/static/
XSendFilePath /data/git/flamenco/flamenco/static/
XsendFilePath /data/git/pillar-svnman/svnman/static/
XsendFilePath /data/git/blender-cloud/static/
XsendFilePath /data/git/blender-cloud/cloud/static/
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
# Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the loglevel for particular
# modules, e.g.
# LogLevel info ssl:warn
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
WSGIDaemonProcess cloud processes=2 threads=64 maximum-requests=10000
WSGIPassAuthorization On
WSGIScriptAlias / /data/git/blender-cloud/runserver.wsgi \
process-group=cloud application-group=%{GLOBAL}
<Directory /data/git/blender-cloud>
<Files runserver.wsgi>
Require all granted
</Files>
</Directory>
# Temporary edit to remap the old cloudapi.blender.org to cloud.blender.org/api
RewriteEngine On
RewriteCond "%{HTTP_HOST}" "^cloudapi\.blender\.org" [NC]
RewriteRule (.*) /api$1 [PT]
# Redirects for blender-cloud projects
RewriteRule "^/p/blender-cloud/?$" "/blog" [R=301,L]
RewriteRule "^/agent327/?$" "/p/agent-327" [R=301,L]
RewriteRule "^/caminandes/?$" "/p/caminandes-3" [R=301,L]
RewriteRule "^/cf2/?$" "/p/creature-factory-2" [R=301,L]
RewriteRule "^/characters/?$" "/p/characters" [R=301,L]
RewriteRule "^/gallery/?$" "/p/gallery" [R=301,L]
RewriteRule "^/hdri/?$" "/p/hdri" [R=301,L]
RewriteRule "^/textures/?$" "/p/textures" [R=301,L]
RewriteRule "^/training/?$" "/courses" [R=301,L]
RewriteRule "^/spring/?$" "/p/spring" [R=301,L]
RewriteRule "^/hero/?$" "/p/hero" [R=301,L]
# Waking the forest was moved from the art gallery to its own workshop
RewriteRule "^/p/gallery/58cfec4f88ac8f1440aeb309/?$" "/p/waking-the-forest" [R=301,L]
</VirtualHost>

View File

@@ -1,21 +0,0 @@
/var/log/apache2/*.log {
daily
missingok
rotate 14
size 100M
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if /etc/init.d/apache2 status > /dev/null ; then \
/etc/init.d/apache2 reload > /dev/null; \
fi;
endscript
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi; \
endscript
}

View File

@@ -1,122 +0,0 @@
<IfModule mod_wsgi.c>
#This config file is provided to give an overview of the directives,
#which are only allowed in the 'server config' context.
#For a detailed description of all avaiable directives please read
#http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives
#WSGISocketPrefix: Configure directory to use for daemon sockets.
#
#Apache's DEFAULT_REL_RUNTIMEDIR should be the proper place for WSGI's
#Socket. In case you want to mess with the permissions of the directory,
#you need to define WSGISocketPrefix to an alternative directory.
#See http://code.google.com/p/modwsgi/wiki/ConfigurationIssues for more
#information
#WSGISocketPrefix /var/run/apache2/wsgi
#WSGIPythonOptimize: Enables basic Python optimisation features.
#
#Sets the level of Python compiler optimisations. The default is '0'
#which means no optimisations are applied.
#Setting the optimisation level to '1' or above will have the effect
#of enabling basic Python optimisations and changes the filename
#extension for compiled (bytecode) files from .pyc to .pyo.
#When the optimisation level is set to '2', doc strings will not be
#generated and retained. This will result in a smaller memory footprint,
#but may cause some Python packages which interrogate doc strings in some
#way to fail.
#WSGIPythonOptimize 0
#WSGIPythonPath: Additional directories to search for Python modules,
# overriding the PYTHONPATH environment variable.
#
#Used to specify additional directories to search for Python modules.
#If multiple directories are specified they should be separated by a ':'.
WSGIPythonPath /opt/python/lib/python3.6/site-packages
#WSGIPythonEggs: Directory to use for Python eggs cache.
#
#Used to specify the directory to be used as the Python eggs cache
#directory for all sub interpreters created within embedded mode.
#This directive achieves the same affect as having set the
#PYTHON_EGG_CACHE environment variable.
#Note that the directory specified must exist and be writable by the user
#that the Apache child processes run as. The directive only applies to
#mod_wsgi embedded mode. To set the Python eggs cache directory for
#mod_wsgi daemon processes, use the 'python-eggs' option to the
#WSGIDaemonProcess directive instead.
#WSGIPythonEggs directory
#WSGIRestrictEmbedded: Enable restrictions on use of embedded mode.
#
#The WSGIRestrictEmbedded directive determines whether mod_wsgi embedded
#mode is enabled or not. If set to 'On' and the restriction on embedded
#mode is therefore enabled, any attempt to make a request against a
#WSGI application which hasn't been properly configured so as to be
#delegated to a daemon mode process will fail with a HTTP internal server
#error response.
#WSGIRestrictEmbedded On|Off
#WSGIRestrictStdin: Enable restrictions on use of STDIN.
#WSGIRestrictStdout: Enable restrictions on use of STDOUT.
#WSGIRestrictSignal: Enable restrictions on use of signal().
#
#Well behaved WSGI applications neither should try to read/write from/to
#STDIN/STDOUT, nor should they try to register signal handlers. If your
#application needs an exception from this rule, you can disable the
#restrictions here.
#WSGIRestrictStdin On
#WSGIRestrictStdout On
#WSGIRestrictSignal On
#WSGIAcceptMutex: Specify type of accept mutex used by daemon processes.
#
#The WSGIAcceptMutex directive sets the method that mod_wsgi will use to
#serialize multiple daemon processes in a process group accepting requests
#on a socket connection from the Apache child processes. If this directive
#is not defined then the same type of mutex mechanism as used by Apache for
#the main Apache child processes when accepting connections from a client
#will be used. If set the method types are the same as for the Apache
#AcceptMutex directive.
#WSGIAcceptMutex default
#WSGIImportScript: Specify a script file to be loaded on process start.
#
#The WSGIImportScript directive can be used to specify a script file to be
#loaded when a process starts. Options must be provided to indicate the
#name of the process group and the application group into which the script
#will be loaded.
#WSGIImportScript process-group=name application-group=name
#WSGILazyInitialization: Enable/disable lazy initialisation of Python.
#
#The WSGILazyInitialization directives sets whether or not the Python
#interpreter is preinitialised within the Apache parent process or whether
#lazy initialisation is performed, and the Python interpreter only
#initialised in the Apache server processes or mod_wsgi daemon processes
#after they have forked from the Apache parent process.
#WSGILazyInitialization On|Off
</IfModule>

View File

@@ -1 +0,0 @@
LoadModule wsgi_module /opt/python/mod-wsgi/mod_wsgi.so

View File

@@ -1,9 +0,0 @@
bash docker-entrypoint.sh
env | sort
apache2ctl start
apache2ctl graceful
/manage.sh operations worker -- -C
celery status --broker amqp://guest:guest@rabbit:5672//
celery events --broker amqp://guest:guest@rabbit:5672//
tail -n 40 -f /var/log/apache2/access.log
tail -n 40 -f /var/log/apache2/error.log

View File

@@ -1,5 +0,0 @@
#!/bin/bash -e
docker build -t armadillica/blender_cloud:latest .
echo "Done, built armadillica/blender_cloud:latest"

View File

@@ -1,6 +0,0 @@
#!/usr/bin/env bash
source /install_scripts.sh
source /manage.sh celery beat -- \
--schedule /data/storage/pillar/celerybeat-schedule.db \
--pid /data/storage/pillar/celerybeat.pid

View File

@@ -1,4 +0,0 @@
#!/usr/bin/env bash
source /install_scripts.sh
source /manage.sh celery worker -- -C

View File

@@ -1,132 +0,0 @@
import os
from collections import defaultdict
DEBUG = False
SCHEME = 'https'
PREFERRED_URL_SCHEME = 'https'
SERVER_NAME = 'cloud.blender.org'
# os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = 'true'
os.environ['PILLAR_MONGO_DBNAME'] = 'cloud'
os.environ['PILLAR_MONGO_PORT'] = '27017'
os.environ['PILLAR_MONGO_HOST'] = 'mongo'
USE_X_SENDFILE = True
STORAGE_BACKEND = 'gcs'
CDN_SERVICE_DOMAIN = 'blendercloud-pro.r.worldssl.net'
CDN_CONTENT_SUBFOLDER = ''
CDN_STORAGE_ADDRESS = 'push-11.cdnsun.com'
CACHE_TYPE = 'redis' # null
CACHE_KEY_PREFIX = 'pw_'
CACHE_REDIS_HOST = 'redis'
CACHE_REDIS_PORT = '6379'
CACHE_REDIS_URL = 'redis://redis:6379'
PILLAR_SERVER_ENDPOINT = 'https://cloud.blender.org/api/'
BLENDER_ID_ENDPOINT = 'https://www.blender.org/id/'
GCLOUD_APP_CREDENTIALS = '/data/config/google_app.json'
GCLOUD_PROJECT = 'blender-cloud'
MAIN_PROJECT_ID = '563a9c8cf0e722006ce97b03'
# MAIN_PROJECT_ID = '57aa07c088bef606e89078bd'
ALGOLIA_INDEX_USERS = 'pro_Users'
ALGOLIA_INDEX_NODES = 'pro_Nodes'
ZENCODER_NOTIFICATIONS_URL = 'https://cloud.blender.org/api/encoding/zencoder/notifications'
FILE_LINK_VALIDITY = defaultdict(
lambda: 3600 * 24 * 30, # default of 1 month.
gcs=3600 * 23, # 23 hours for Google Cloud Storage.
cdnsun=3600 * 23
)
LOGGING = {
'version': 1,
'formatters': {
'default': {'format': '%(levelname)8s %(name)s %(message)s'}
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'default',
'stream': 'ext://sys.stderr',
}
},
'loggers': {
'pillar': {'level': 'INFO'},
# 'pillar.auth': {'level': 'DEBUG'},
# 'pillar.api.blender_id': {'level': 'DEBUG'},
# 'pillar.api.blender_cloud.subscription': {'level': 'DEBUG'},
'bcloud': {'level': 'INFO'},
'cloud': {'level': 'INFO'},
'attract': {'level': 'INFO'},
'flamenco': {'level': 'INFO'},
# 'pillar.api.file_storage': {'level': 'DEBUG'},
# 'pillar.api.file_storage.ensure_valid_link': {'level': 'INFO'},
'pillar.api.file_storage.refresh_links_for_backend': {'level': 'DEBUG'},
'werkzeug': {'level': 'DEBUG'},
'eve': {'level': 'WARNING'},
# 'elasticsearch': {'level': 'DEBUG'},
},
'root': {
'level': 'WARNING',
'handlers': [
'console',
],
}
}
# Latest version of the add-on.
BLENDER_CLOUD_ADDON_VERSION = '1.9.0'
REDIRECTS = {
# For old links, refer to the services page (hopefully it refreshes then)
'downloads/blender_cloud-latest-bundle.zip': 'https://cloud.blender.org/services#blender-addon',
# Latest Blender Cloud add-on.
'downloads/blender_cloud-latest-addon.zip':
f'https://storage.googleapis.com/institute-storage/addons/'
f'blender_cloud-{BLENDER_CLOUD_ADDON_VERSION}.addon.zip',
# Redirect old Grafista endpoint to /stats
'/stats/': '/stats',
}
UTM_LINKS = {
'cartoon_brew': {
'image': 'https://imgur.com/13nQTi3.png',
'link': 'https://store.blender.org/product/membership/'
}
}
# Disabled until we have regenerated the majority of the links.
CELERY_BEAT_SCHEDULE = {
'regenerate-expired-links': {
'task': 'pillar.celery.file_link_tasks.regenerate_all_expired_links',
'schedule': 600, # every N seconds
'args': ('gcs', 500)
},
}
SVNMAN_REPO_URL = 'https://svn.blender.cloud/repo/'
SVNMAN_API_URL = 'https://svn.blender.cloud/api/'
# Mail options, see pillar.celery.email_tasks.
SMTP_HOST = 'proog.blender.org'
SMTP_PORT = 25
SMTP_USERNAME = 'server@blender.cloud'
SMTP_TIMEOUT = 30 # timeout in seconds, https://docs.python.org/3/library/smtplib.html#smtplib.SMTP
MAIL_RETRY = 180 # in seconds, delay until trying to send an email again.
MAIL_DEFAULT_FROM_NAME = 'Blender Cloud'
MAIL_DEFAULT_FROM_ADDR = 'cloudsupport@blender.org'
# MUST be 8 characters long, see pillar.flask_extra.HashedPathConverter
# STATIC_FILE_HASH = '12345678'
# The value used in production is appended from Dockerfile.

View File

@@ -1,15 +0,0 @@
#!/usr/bin/env bash
source /install_scripts.sh
# Make sure that log rotation works.
mkdir -p ${APACHE_LOG_DIR}
service cron start
if [ "$DEV" = "true" ]; then
echo "Running in development mode"
cd /data/git/blender-cloud
exec bash /manage.sh runserver --host='0.0.0.0'
else
exec /usr/sbin/apache2ctl -D FOREGROUND
fi

View File

@@ -1,21 +0,0 @@
#!/bin/sh
if [ -f /installed ]; then
return
fi
SITEPKG=$(echo /opt/python/lib/python3.*/site-packages)
echo "Installing Blender Cloud packages into $SITEPKG"
# TODO: 'pip3 install -e' runs 'setup.py develop', which runs 'setup.py egg_info',
# which can't write the egg info to the read-only /data/git volume. This is why
# we manually install the links.
for SUBPROJ in /data/git/{pillar,pillar-python-sdk,attract,flamenco,pillar-svnman}; do
NAME=$(python3 $SUBPROJ/setup.py --name)
echo "... $NAME"
echo $SUBPROJ >> $SITEPKG/easy-install.pth
echo $SUBPROJ > $SITEPKG/$NAME.egg-link
done
echo "All packages installed."
touch /installed

View File

@@ -1,5 +0,0 @@
#!/usr/bin/env bash
set -e
cd /data/git/blender-cloud
exec python manage.py "$@"

View File

@@ -1,96 +0,0 @@
# Setting up a production machine
To get the docker stack up and running, we use the following, on an Ubuntu 16.10 machine.
## 0. Basic stuff
Install the machine, use `locale-gen nl_NL.UTF-8` or similar commands to generate locale
definitions. Set up automatic security updates and backups, the usual.
## 1. Install Docker
Install Docker itself, as described in the
[Docker CE for Ubuntu manual](https://store.docker.com/editions/community/docker-ce-server-ubuntu?tab=description):
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable"
apt-get update
apt-get install docker-ce
## 2. Configure Docker to use "overlay"
Configure Docker to use "overlay" instead of "aufs" for the images. This prevents
[segfaults in auplink](https://bugs.launchpad.net/ubuntu/+source/aufs-tools/+bug/1442568).
1. Set `DOCKER_OPTS="-s overlay"` in `/etc/defaults/docker`
2. Copy `/lib/systemd/system/docker.service` to `/etc/systemd/system/docker.service`.
This allows later upgrading of docker without overwriting the changes we're about to do.
2. Edit the `[Service]` section of `/etc/systemd/system/docker.service`:
1. Add `EnvironmentFile=/etc/default/docker`
2. Append ` $DOCKER_OPTS` to the `ExecStart` line
3. Run `systemctl daemon-reload`
4. Remove all your containers and images.
5. Restart Docker: `systemctl restart docker`
## 3. Pull the Blender Cloud docker image
`docker pull armadillica/blender_cloud:latest`
## 4. Get docker-compose + our repositories
See the [Quick setup](../README.md) on how to get those. Then run:
cd /data/git/blender-cloud/docker
docker-compose up -d
Set up permissions for Docker volumes; the following should be writable by
- `/data/storage/pillar`: writable by `www-data` and `root` (do a `chown root:www-data`
and `chmod 2770`).
- `/data/storage/db`: writable by uid 999.
## 5. Set up TLS
Place TLS certificates in `/data/certs/{cloud,cloudapi}.blender.org.pem`.
They should contain (in order) the private key, the host certificate, and the
CA certificate.
## 6. Create a local config
Blender Cloud expects the following files to exist:
- `/data/git/blender_cloud/config_local.py` with machine-local configuration overrides
- `/data/config/google_app.json` with Google Cloud Storage credentials.
When run from Docker, the `docker/4_run/config_local.py` file will be used. Overrides for that file
can be placed in `/data/config/config_secrets.py`.
## 7. ElasticSearch & kibana
ElasticSearch and Kibana run in our self-rolled images. This is needed because by default
- ElasticSearch uses up to 2 GB of RAM, which is too much for our droplet, and
- the Docker images contain the proprietary X-Pack plugin, which we don't want.
This also gives us the opportunity to let Kibana do its optimization when we build the image, rather
than every time the container is recreated.
`/data/storage/elasticsearch` needs to be writable by UID 1000, GID 1000.
Kibana connects to [ElasticProxy](https://github.com/armadillica/elasticproxy), which only allows
GET, HEAD, and some specific POST requests. This ensures that the public-facing Kibana cannot be
used to change the ElasticSearch database.
Production Kibana can be placed in read-only mode, but this is not necessary now that we use
ElasticProxy. However, I've left this in here as reference.
`curl -XPUT 'localhost:9200/.kibana/_settings' -d '{ "index.blocks.read_only" : true }'`
If editing is desired, temporarily turn off read-only mode:
`curl -XPUT 'localhost:9200/.kibana/_settings' -d '{ "index.blocks.read_only" : false }'`

16
docker/build.sh Executable file
View File

@@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -x;
set -e;
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cd $DIR;
cd 1_base/;
bash build.sh;
cd ../2_build/;
bash build.sh;
cd ../3_run/;
bash build.sh;

View File

@@ -1,191 +1,68 @@
version: '3.4' mongo:
services: image: mongo
mongo: container_name: mongo
image: mongo:3.4.2 restart: always
container_name: mongo volumes:
restart: always - /data/storage/db:/data/db
volumes: ports:
- /data/storage/db:/data/db - "127.0.0.1:27017:27017"
- /data/storage/db-bak:/data/db-bak # for backing up stuff etc. redis:
ports: image: redis
- "127.0.0.1:27017:27017" container_name: redis
logging: restart: always
driver: "json-file" blender_cloud:
options: image: armadillica/blender_cloud
max-size: "200k" container_name: blender_cloud
max-file: "20" restart: always
environment:
redis: VIRTUAL_HOST: http://cloudapi.blender.org/*,https://cloudapi.blender.org/*,http://cloud.blender.org/*,https://cloud.blender.org/*,http://pillar-web/*
image: redis:3.2.8 VIRTUAL_HOST_WEIGHT: 10
container_name: redis FORCE_SSL: "true"
restart: always volumes:
ports: - /data/git/blender-cloud:/data/git/blender-cloud:ro
- "127.0.0.1:6379:6379" - /data/git/attract:/data/git/attract:ro
logging: - /data/git/flamenco:/data/git/flamenco:ro
driver: "json-file" - /data/git/pillar:/data/git/pillar:ro
options: - /data/git/pillar-python-sdk:/data/git/pillar-python-sdk:ro
max-size: "200k" - /data/config:/data/config:ro
max-file: "20" - /data/storage/pillar:/data/storage/pillar
links:
rabbit: - mongo
image: rabbitmq:3.6.10 - redis
container_name: rabbit # notifserv:
restart: always # container_name: notifserv
ports: # image: armadillica/pillar-notifserv:cd8fa678436563ac3b800b2721e36830c32e4656
- "127.0.0.1:5672:5672" # restart: always
logging: # links:
driver: "json-file" # - mongo
options: # environment:
max-size: "200k" # VIRTUAL_HOST: https://cloud.blender.org/notifications*,http://pillar-web/notifications*
max-file: "20" # VIRTUAL_HOST_WEIGHT: 20
# FORCE_SSL: true
elastic: grafista:
image: armadillica/elasticsearch:6.1.1 image: armadillica/grafista
container_name: elastic container_name: grafista
restart: always restart: always
volumes: environment:
# NOTE: this path must be writable by UID=1000 GID=1000. VIRTUAL_HOST: http://cloud.blender.org/stats/*,https://cloud.blender.org/stats/*,http://blender-cloud/stats/*
- /data/storage/elastic:/usr/share/elasticsearch/data VIRTUAL_HOST_WEIGHT: 20
ports: FORCE_SSL: "true"
- "127.0.0.1:9200:9200" volumes:
environment: - /data/git/grafista:/data/git/grafista:ro
ES_JAVA_OPTS: "-Xms256m -Xmx256m" - /data/storage/grafista:/data/storage
logging: haproxy:
driver: "json-file" image: dockercloud/haproxy
options: container_name: haproxy
max-size: "200k" restart: always
max-file: "20" ports:
- "443:443"
elasticproxy: - "80:80"
image: armadillica/elasticproxy:1.2 environment:
container_name: elasticproxy - CERT_FOLDER=/certs/
restart: always - TIMEOUT=connect 5s, client 5m, server 10m
command: /elasticproxy -elastic http://elastic:9200/ links:
depends_on: - blender_cloud
- elastic - grafista
logging: # - notifserv
driver: "json-file" volumes:
options: - '/data/certs:/certs'
max-size: "200k"
max-file: "20"
kibana:
image: armadillica/kibana:6.1.1
container_name: kibana
restart: always
environment:
SERVER_NAME: "stats.cloud.blender.org"
ELASTICSEARCH_URL: http://elasticproxy:9200
CONSOLE_ENABLED: 'false'
VIRTUAL_HOST: http://stats.cloud.blender.org/*,https://stats.cloud.blender.org/*,http://stats.cloud.local/*,https://stats.cloud.local/*
VIRTUAL_HOST_WEIGHT: 20
FORCE_SSL: "true"
# See https://github.com/elastic/kibana/issues/5170#issuecomment-163042525
NODE_OPTIONS: "--max-old-space-size=200"
depends_on:
- elasticproxy
logging:
driver: "json-file"
options:
max-size: "200k"
max-file: "20"
blender_cloud:
image: armadillica/blender_cloud:latest
container_name: blender_cloud
restart: always
environment:
VIRTUAL_HOST: http://cloud.blender.org/*,https://cloud.blender.org/*,http://cloud.local/*,https://cloud.local/*
VIRTUAL_HOST_WEIGHT: 10
FORCE_SSL: "true"
GZIP_COMPRESSION_TYPE: "text/html text/plain text/css application/javascript"
PILLAR_CONFIG: /data/config/config_secrets.py
volumes:
# format: HOST:CONTAINER
- /data/config:/data/config:ro
- /data/storage/pillar:/data/storage/pillar
- /data/log:/var/log
depends_on:
- mongo
- redis
- rabbit
celery_worker:
image: armadillica/blender_cloud:latest
entrypoint: /celery-worker.sh
container_name: celery_worker
restart: always
environment:
PILLAR_CONFIG: /data/config/config_secrets.py
volumes:
# format: HOST:CONTAINER
- /data/config:/data/config:ro
- /data/storage/pillar:/data/storage/pillar
- /data/log:/var/log
depends_on:
- mongo
- redis
- rabbit
logging:
driver: "json-file"
options:
max-size: "200k"
max-file: "20"
celery_beat:
image: armadillica/blender_cloud:latest
entrypoint: /celery-beat.sh
container_name: celery_beat
restart: always
environment:
PILLAR_CONFIG: /data/config/config_secrets.py
volumes:
# format: HOST:CONTAINER
- /data/config:/data/config:ro
- /data/storage/pillar:/data/storage/pillar
- /data/log:/var/log
depends_on:
- mongo
- redis
- rabbit
- celery_worker
logging:
driver: "json-file"
options:
max-size: "200k"
max-file: "20"
letsencrypt:
image: armadillica/picohttp:1.0
container_name: letsencrypt
restart: always
environment:
WEBROOT: /data/letsencrypt
LISTEN: '[::]:80'
VIRTUAL_HOST: http://cloud.blender.org/.well-known/*, http://stats.cloud.blender.org/.well-known/*
VIRTUAL_HOST_WEIGHT: 30
volumes:
- /data/letsencrypt:/data/letsencrypt
haproxy:
image: dockercloud/haproxy:1.5.3
container_name: haproxy
restart: always
ports:
- "443:443"
- "80:80"
environment:
- ADDITIONAL_SERVICES=docker:blender_cloud,docker:letsencrypt,docker:kibana
- CERT_FOLDER=/certs/
- TIMEOUT=connect 5s, client 5m, server 10m
- SSL_BIND_CIPHERS=ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
- SSL_BIND_OPTIONS=no-sslv3
- EXTRA_GLOBAL_SETTINGS=tune.ssl.default-dh-param 2048
depends_on:
- blender_cloud
- letsencrypt
- kibana
volumes:
- '/data/certs:/certs'
- /var/run/docker.sock:/var/run/docker.sock

View File

@@ -1,10 +0,0 @@
FROM docker.elastic.co/elasticsearch/elasticsearch:6.1.1
LABEL maintainer Sybren A. Stüvel <sybren@blender.studio>
RUN elasticsearch-plugin remove --purge x-pack
ADD elasticsearch.yml jvm.options /usr/share/elasticsearch/config/
USER root
RUN chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/config/
USER elasticsearch

View File

@@ -1,6 +0,0 @@
FROM docker.elastic.co/kibana/kibana:6.1.1
LABEL maintainer Sybren A. Stüvel <sybren@blender.studio>
RUN bin/kibana-plugin remove x-pack
ADD kibana.yml /usr/share/kibana/config/kibana.yml
RUN kibana 2>&1 | grep -m 1 "Optimization of .* complete"

View File

@@ -1,15 +0,0 @@
#!/bin/bash -e
# When updating this, also update the versions in Dockerfile-*, and make sure that
# it matches the versions of the elasticsearch and elasticsearch_dsl packages
# used in Pillar. Those don't have to match exactly, but the major version should.
VERSION=6.1.1
docker build -t armadillica/elasticsearch:${VERSION} -f Dockerfile-elastic .
docker build -t armadillica/kibana:${VERSION} -f Dockerfile-kibana .
docker tag armadillica/elasticsearch:${VERSION} armadillica/elasticsearch:latest
docker tag armadillica/kibana:${VERSION} armadillica/kibana:latest
echo "Done, built armadillica/elasticsearch:${VERSION} and armadillica/kibana:${VERSION}"
echo "Also tagged as armadillica/elasticsearch:latest and armadillica/kibana:latest"

View File

@@ -1,7 +0,0 @@
cluster.name: "blender-cloud"
network.host: 0.0.0.0
# minimum_master_nodes need to be explicitly set when bound on a public IP
# set to 1 to allow single node clusters
# Details: https://github.com/elastic/elasticsearch/pull/17288
discovery.zen.minimum_master_nodes: 1

View File

@@ -1,112 +0,0 @@
## JVM configuration
################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
# Sybren: uncommented so that we can set those options using the ES_JAVA_OPTS environment variable.
#-Xms512m
#-Xmx512m
################################################################
## Expert settings
################################################################
##
## All settings below this section are considered
## expert settings. Don't tamper with them unless
## you understand what you are doing
##
################################################################
## GC configuration
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
## optimizations
# pre-touch memory pages used by the JVM during initialization
-XX:+AlwaysPreTouch
## basic
# force the server VM (remove on 32-bit client JVMs)
-server
# explicitly set the stack size (reduce to 320k on 32-bit client JVMs)
-Xss1m
# set to headless, just in case
-Djava.awt.headless=true
# ensure UTF-8 encoding by default (e.g. filenames)
-Dfile.encoding=UTF-8
# use our provided JNA always versus the system one
-Djna.nosys=true
# use old-style file permissions on JDK9
-Djdk.io.permissionsUseCanonicalPath=true
# flags to configure Netty
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0
# log4j 2
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-Dlog4j.skipJansi=true
## heap dumps
# generate a heap dump when an allocation from the Java heap fails
# heap dumps are created in the working directory of the JVM
-XX:+HeapDumpOnOutOfMemoryError
# specify an alternative path for heap dumps
# ensure the directory exists and has sufficient space
#-XX:HeapDumpPath=${heap.dump.path}
## GC logging
#-XX:+PrintGCDetails
#-XX:+PrintGCTimeStamps
#-XX:+PrintGCDateStamps
#-XX:+PrintClassHistogram
#-XX:+PrintTenuringDistribution
#-XX:+PrintGCApplicationStoppedTime
# log GC status to a file with time stamps
# ensure the directory exists
#-Xloggc:${loggc}
# By default, the GC log file will not rotate.
# By uncommenting the lines below, the GC log file
# will be rotated every 128MB at most 32 times.
#-XX:+UseGCLogFileRotation
#-XX:NumberOfGCLogFiles=32
#-XX:GCLogFileSize=128M
# Elasticsearch 5.0.0 will throw an exception on unquoted field names in JSON.
# If documents were already indexed with unquoted fields in a previous version
# of Elasticsearch, some operations may throw errors.
#
# WARNING: This option will be removed in Elasticsearch 6.0.0 and is provided
# only for migration purposes.
#-Delasticsearch.json.allow_unquoted_field_names=true

View File

@@ -1,8 +0,0 @@
---
server.name: kibana
server.host: "0"
elasticsearch.url: http://elasticsearch:9200
# Hide dev tools
console.enabled: false

View File

@@ -1,17 +0,0 @@
#!/usr/bin/env bash
set -xe
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cd $DIR/1_base
bash build.sh
cd $DIR/2_buildpy
bash build.sh
cd $DIR/3_buildwheels
bash build.sh
cd $DIR/4_run
bash build.sh

View File

@@ -1,5 +0,0 @@
# Change to suit your needs, then place in /etc/cron.d/mongo-backup
# (so remove the .cron from the name)
MAILTO=yourname@youraddress.org
30 5 * * * root /root/docker/mongo-backup.sh

View File

@@ -1,28 +0,0 @@
#!/bin/bash -e
BACKUPDIR=/data/storage/db-bak
DATE=$(date +'%Y-%m-%d-%H%M')
ARCHIVE=$BACKUPDIR/mongo-live-$DATE.tar.xz
# Just a sanity check before we give it to 'rm -rf'
if [ -z "$DATE" ]; then
echo "Empty string found where the date should be, aborting."
exit 1
fi
# /data/db-bak in Docker is /data/storage/db-bak on the host.
docker exec mongo mongodump -d cloud \
--out /data/db-bak/dump-$DATE \
--excludeCollection tokens \
--excludeCollection flamenco_task_logs \
--quiet
cd $BACKUPDIR
tar -Jcf $ARCHIVE dump-$DATE/
rm -rf dump-$DATE
TO_DELETE="$(ls $BACKUPDIR/mongo-live-*.tar.xz | head -n -7)"
[ -z "$TO_DELETE" ] || rm "$TO_DELETE"
rsync -a $BACKUPDIR/mongo-live-*.tar.xz cloud-backup@swami-direct.blender.cloud:

View File

@@ -1,27 +0,0 @@
#!/bin/bash -e
# First time creating a certificate for a domain, use:
# certbot certonly --webroot -w /data/letsencrypt -d $DOMAINNAME
cd /data/letsencrypt
certbot renew
echo
echo "Recreating HAProxy certificates"
for certdir in /etc/letsencrypt/live/*; do
domain=$(basename $certdir)
echo " - $domain"
cat $certdir/privkey.pem $certdir/fullchain.pem > $domain.pem
mv $domain.pem /data/certs/
done
echo
echo -n "Restarting "
docker restart haproxy
echo "Certificate renewal completed."

31
gulp
View File

@@ -1,31 +0,0 @@
#!/bin/bash -ex
GULP=./node_modules/.bin/gulp
function install() {
npm install
touch $GULP # installer doesn't always touch this after a build, so we do.
}
# Rebuild Gulp if missing or outdated.
[ -e $GULP ] || install
[ gulpfile.js -nt $GULP ] && install
if [ "$1" == "watch" ]; then
# Treat "gulp watch" as "gulp && gulp watch"
$GULP
elif [ "$1" == "all" ]; then
# This is useful when building the Blender Cloud project for the first time.
# Run "gulp" once inside the repo
$GULP
# Run ./gulp in all depending projects (pillar, attract, flamenco, pillar-svnman)
declare -a repos=("pillar" "attract" "flamenco" "pillar-svnman")
for r in "${repos[@]}"
do
cd ../$r
./gulp
done
exit 1
fi
exec $GULP "$@"

View File

@@ -1,160 +0,0 @@
var argv = require('minimist')(process.argv.slice(2));
var autoprefixer = require('gulp-autoprefixer');
var cache = require('gulp-cached');
var chmod = require('gulp-chmod');
var concat = require('gulp-concat');
var git = require('gulp-git');
var gulp = require('gulp');
var gulpif = require('gulp-if');
var pug = require('gulp-pug');
var livereload = require('gulp-livereload');
var plumber = require('gulp-plumber');
var rename = require('gulp-rename');
var sass = require('gulp-sass');
var sourcemaps = require('gulp-sourcemaps');
var uglify = require('gulp-uglify-es').default;
var enabled = {
uglify: argv.production,
maps: !argv.production,
failCheck: !argv.production,
prettyPug: !argv.production,
cachify: !argv.production,
cleanup: argv.production,
chmod: argv.production,
};
var destination = {
css: 'cloud/static/assets/css',
pug: 'cloud/templates',
js: 'cloud/static/assets/js',
}
var source = {
pillar: '../pillar/',
bootstrap: 'node_modules/bootstrap/',
popper: 'node_modules/popper.js/'
}
/* CSS */
gulp.task('styles', function() {
gulp.src('src/styles/**/*.sass')
.pipe(gulpif(enabled.failCheck, plumber()))
.pipe(gulpif(enabled.maps, sourcemaps.init()))
.pipe(sass({
outputStyle: 'compressed'}
))
.pipe(autoprefixer("last 3 versions"))
.pipe(gulpif(enabled.maps, sourcemaps.write(".")))
.pipe(gulp.dest(destination.css))
.pipe(gulpif(argv.livereload, livereload()));
});
/* Templates - Pug */
gulp.task('templates', function() {
gulp.src('src/templates/**/*.pug')
.pipe(gulpif(enabled.failCheck, plumber()))
.pipe(gulpif(enabled.cachify, cache('templating')))
.pipe(pug({
pretty: enabled.prettyPug
}))
.pipe(gulp.dest(destination.pug))
.pipe(gulpif(argv.livereload, livereload()));
// TODO(venomgfx): please check why 'gulp watch' doesn't pick up on .txt changes.
gulp.src('src/templates/**/*.txt')
.pipe(gulpif(enabled.failCheck, plumber()))
.pipe(gulpif(enabled.cachify, cache('templating')))
.pipe(gulp.dest(destination.pug))
.pipe(gulpif(argv.livereload, livereload()));
});
/* Individual Uglified Scripts */
gulp.task('scripts', function() {
gulp.src('src/scripts/*.js')
.pipe(gulpif(enabled.failCheck, plumber()))
.pipe(gulpif(enabled.cachify, cache('scripting')))
.pipe(gulpif(enabled.maps, sourcemaps.init()))
.pipe(gulpif(enabled.uglify, uglify()))
.pipe(rename({suffix: '.min'}))
.pipe(gulpif(enabled.maps, sourcemaps.write(".")))
.pipe(gulpif(enabled.chmod, chmod(644)))
.pipe(gulp.dest(destination.js))
.pipe(gulpif(argv.livereload, livereload()));
});
/* Collection of scripts in src/scripts/tutti/ to merge into tutti.min.js */
/* Since it's always loaded, it's only for functions that we want site-wide */
gulp.task('scripts_concat_tutti', function() {
gulp.src('src/scripts/tutti/**/*.js')
.pipe(gulpif(enabled.failCheck, plumber()))
.pipe(gulpif(enabled.maps, sourcemaps.init()))
.pipe(concat("tutti.min.js"))
.pipe(gulpif(enabled.uglify, uglify()))
.pipe(gulpif(enabled.maps, sourcemaps.write(".")))
.pipe(gulpif(enabled.chmod, chmod(644)))
.pipe(gulp.dest(destination.js))
.pipe(gulpif(argv.livereload, livereload()));
});
// Combine all needed Bootstrap JavaScript into a single file.
gulp.task('scripts_concat_bootstrap', function() {
toUglify = [
source.popper + 'dist/umd/popper.min.js',
source.bootstrap + 'js/dist/index.js',
source.bootstrap + 'js/dist/util.js',
source.bootstrap + 'js/dist/tooltip.js',
source.bootstrap + 'js/dist/dropdown.js',
];
gulp.src(toUglify)
.pipe(gulpif(enabled.failCheck, plumber()))
.pipe(gulpif(enabled.maps, sourcemaps.init()))
.pipe(concat("bootstrap.min.js"))
.pipe(gulpif(enabled.uglify, uglify()))
.pipe(gulpif(enabled.maps, sourcemaps.write(".")))
.pipe(gulpif(enabled.chmod, chmod(644)))
.pipe(gulp.dest(destination.js))
.pipe(gulpif(argv.livereload, livereload()));
});
// While developing, run 'gulp watch'
gulp.task('watch',function() {
// Only listen for live reloads if ran with --livereload
if (argv.livereload){
livereload.listen();
}
gulp.watch('src/styles/**/*.sass',['styles']);
gulp.watch(source.pillar + 'src/styles/**/*.sass',['styles']);
gulp.watch('src/templates/**/*.pug',['templates']);
gulp.watch('src/scripts/*.js',['scripts']);
gulp.watch('src/scripts/tutti/**/*.js',['scripts_concat_tutti']);
});
// Erases all generated files in output directories.
gulp.task('cleanup', function() {
var paths = [];
for (attr in destination) {
paths.push(destination[attr]);
}
git.clean({ args: '-f -X ' + paths.join(' ') }, function (err) {
if(err) throw err;
});
});
// Run 'gulp' to build everything at once
var tasks = [];
if (enabled.cleanup) tasks.push('cleanup');
gulp.task('default', tasks.concat(['styles', 'templates', 'scripts', 'scripts_concat_tutti', 'scripts_concat_bootstrap']));

View File

@@ -1,7 +1,40 @@
#!/usr/bin/env python #!/usr/bin/env python
from __future__ import print_function
import logging
from flask import current_app
from pillar import cli from pillar import cli
from runserver import app from pillar.cli import manager_maintenance
from cloud import app
log = logging.getLogger(__name__)
@manager_maintenance.command
def reconcile_subscribers():
"""For every user, check their subscription status with the store."""
from pillar.auth.subscriptions import fetch_user
users_coll = current_app.data.driver.db['users']
unsubscribed_users = []
for user in users_coll.find({'roles': 'subscriber'}):
print('Processing %s' % user['email'])
print(' Checking subscription')
user_store = fetch_user(user['email'])
if user_store['cloud_access'] == 0:
print(' Removing subscriber role')
users_coll.update(
{'_id': user['_id']},
{'$pull': {'roles': 'subscriber'}})
unsubscribed_users.append(user['email'])
if not unsubscribed_users:
return
print('The following users have been unsubscribed')
for user in unsubscribed_users:
print(user)
cli.manager.app = app cli.manager.app = app
cli.manager.run() cli.manager.run()

5601
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,31 +0,0 @@
{
"name": "blender-cloud",
"license": "GPL-2.0+",
"author": "Blender Institute",
"repository": {
"type": "git",
"url": "git://git.blender.org/blender-cloud.git"
},
"devDependencies": {
"gulp": "~3.9.1",
"gulp-autoprefixer": "~6.0.0",
"gulp-cached": "~1.1.1",
"gulp-chmod": "~2.0.0",
"gulp-concat": "~2.6.1",
"gulp-if": "^2.0.2",
"gulp-git": "~2.8.0",
"gulp-livereload": "~4.0.0",
"gulp-plumber": "~1.2.0",
"gulp-pug": "~4.0.1",
"gulp-rename": "~1.4.0",
"gulp-sass": "~4.0.1",
"gulp-sourcemaps": "~2.6.4",
"gulp-uglify-es": "^1.0.4",
"minimist": "^1.2.0"
},
"dependencies": {
"bootstrap": "^4.1.3",
"jquery": "^3.3.1",
"popper.js": "^1.14.4"
}
}

View File

@@ -1,12 +0,0 @@
-r ../pillar-python-sdk/requirements-dev.txt
-r ../pillar/requirements-dev.txt
-r ../attract/requirements-dev.txt
-r ../flamenco/requirements-dev.txt
-r ../pillar-svnman/requirements-dev.txt
-e ../pillar-python-sdk
-e ../pillar
-e ../attract
-e ../flamenco
-e ../pillar-svnman
-e .

View File

@@ -1,4 +1,67 @@
-r ../pillar/requirements.txt # Primary requirements
-r ../attract/requirements.txt # pillarsdk
-r ../flamenco/requirements.txt # pillar
-r ../pillar-svnman/requirements.txt # attract
# flamenco
# Secondary requirements (i.e. pulled in from primary requirements)
algoliasearch==1.8.0
attrs==16.2.0
bcrypt==2.0.0
blinker==1.4
bugsnag==2.3.1
bleach==1.4.3
Cerberus==0.9.2
cffi==1.7.0
commonmark==0.7.2
cryptography==1.4
enum34==1.1.6
Eve==0.6.3
Events==0.2.1
Flask==0.10.1
Flask-Cache==0.13.1
Flask-Script==2.0.5
Flask-Login==0.3.2
Flask-OAuthlib==0.9.3
Flask-PyMongo==0.4.1
Flask-WTF==0.12
flup==1.0.2
future==0.15.2
gcloud==0.12.0
google-apitools==0.4.11
googleapis-common-protos==1.2.0
html5lib==0.9999999
httplib2==0.9.2
idna==2.0
ipaddress==1.0.16
itsdangerous==0.24
Jinja2==2.8
MarkupSafe==0.23
markdown==2.6.7
ndg-httpsclient==0.4.0
oauth2client==3.0.0
oauthlib==1.1.2
pathlib2==2.2.1
Pillow==2.8.1
protobuf==3.0.0
protorpc==0.11.1
pyasn1==0.1.9
pyasn1-modules==0.0.8
pycparser==2.14
pycrypto==2.6.1
pylru==1.0.4
pymongo==3.3.0
pyOpenSSL==0.15.1
python-dateutil==2.5.3
redis==2.10.5
requests==2.9.1
requests-oauthlib==0.6.2
rsa==3.4.2
scandir==1.4
simplejson==3.8.2
six==1.10.0
svn==0.3.43
WebOb==1.5.0
Werkzeug==0.11.10
WTForms==2.1
zencoder==0.6.5

40
rsync_ui.sh Executable file
View File

@@ -0,0 +1,40 @@
#!/usr/bin/env bash
PILLAR_DIR=$(python <<EOT
from __future__ import print_function
import os.path
import pillar
print(os.path.dirname(os.path.dirname(pillar.__file__)))
EOT
)
ASSETS="$PILLAR_DIR/pillar/web/static/assets/"
TEMPLATES="$PILLAR_DIR/pillar/web/templates/"
if [ ! -d "$ASSETS" ]; then
echo "Unable to find assets dir $ASSETS"
exit 1
fi
cd $PILLAR_DIR
if [ $(git rev-parse --abbrev-ref HEAD) != "production" ]; then
echo "You are NOT on the production branch, refusing to rsync_ui." >&2
exit 1
fi
echo
echo "*** GULPA GULPA ***"
if [ -x ./node_modules/.bin/gulp ]; then
./node_modules/.bin/gulp --production
else
gulp --production
fi
echo
echo "*** SYNCING ASSETS ***"
rsync -avh $ASSETS root@cloud.blender.org:/data/git/pillar/pillar/web/static/assets/
echo
echo "*** SYNCING TEMPLATES ***"
rsync -avh $TEMPLATES root@cloud.blender.org:/data/git/pillar/pillar/web/templates/

View File

@@ -1,23 +1,22 @@
from os.path import abspath, dirname from os.path import abspath, dirname
import sys import sys
my_path = dirname(abspath(__file__)) activate_this = '/data/venv/bin/activate_this.py'
sys.path.append(my_path) execfile(activate_this, dict(__file__=activate_this))
from flup.server.fcgi import WSGIServer
from pillar import PillarServer from pillar import PillarServer
from attract import AttractExtension from attract import AttractExtension
from flamenco import FlamencoExtension from flamenco import FlamencoExtension
from svnman import SVNManExtension
from cloud import CloudExtension sys.path.append('/data/git/blender-cloud/')
attract = AttractExtension() attract = AttractExtension()
flamenco = FlamencoExtension() flamenco = FlamencoExtension()
svnman = SVNManExtension()
cloud = CloudExtension()
application = PillarServer(my_path) application = PillarServer(dirname(abspath(__file__)))
application.load_extension(attract, '/attract') application.load_extension(attract, '/attract')
application.load_extension(flamenco, '/flamenco') application.load_extension(flamenco, '/flamenco')
application.load_extension(svnman, '/svn')
application.load_extension(cloud, None)
application.process_extensions() application.process_extensions()
if __name__ == '__main__':
WSGIServer(application).run()

View File

@@ -1,5 +0,0 @@
[tool:pytest]
addopts = -v --cov cloud --cov-report term-missing --ignore node_modules --ignore docker
[pep8]
max-line-length = 100

View File

@@ -1,109 +0,0 @@
/**
* Support for fetching & rendering assets by tags.
*/
(function($) {
/* How many nodes to load initially, and when clicked on the 'Load Next' link. */
const LOAD_INITIAL_COUNT = 5;
const LOAD_NEXT_COUNT = 3;
/* Renders a node as a <li> element, returns a jQuery object. */
function renderAsset(node) {
let li = $('<li>').addClass('tagged-asset');
let link = $('<a>')
.attr('href', '/nodes/' + node._id + '/redir')
.appendTo(li);
function warnNoPicture() {
li.addClass('warning');
link.text('no picture for node ' + node._id);
}
if (!node.picture) {
warnNoPicture();
return li;
}
// TODO: show 'loading' thingy
$.get('/api/files/' + node.picture)
.fail(function(error) {
let msg = xhrErrorResponseMessage(error);
li.addClass('error').text(msg);
})
.done(function(resp) {
// Render the picture if it has the proper size.
var show_variation = null;
if (typeof resp.variations != 'undefined') {
for (variation of resp.variations) {
if (variation.size != 'm') continue;
show_variation = variation;
break;
}
}
if (show_variation == null) {
warnNoPicture();
return;
}
let img = $('<img>')
.attr('alt', node.name)
.attr('src', variation.link)
.attr('width', variation.width)
.attr('height', variation.height);
link.append(img);
});
return li;
}
function loadNext(ul_element) {
let $ul = $(ul_element);
let tagged_assets = ul_element.tagged_assets; // Stored here by loadTaggedAssets().
let already_loaded = $ul.find('li.tagged-asset').length;
let load_next = $ul.find('li.load-next');
let nodes_to_load = tagged_assets.slice(already_loaded, already_loaded + LOAD_NEXT_COUNT);
for (node of nodes_to_load) {
let li = renderAsset(node);
load_next.before(li);
}
if (already_loaded + LOAD_NEXT_COUNT >= tagged_assets.length)
load_next.remove();
}
$.fn.loadTaggedAssets = function(api_base_url) {
this.each(function(index, ul_element) {
// TODO(Sybren): show a 'loading' animation.
$.get('/api/nodes/tagged/' + ul_element.dataset.assetTag)
.fail(function(error) {
let msg = xhrErrorResponseMessage(error);
$('<li>').addClass('error').text(msg).appendTo(ul_element);
})
.done(function(resp) {
// 'resp' is a list of node documents.
// Store the response on the DOM <ul>-element so that we can later render more.
ul_element.tagged_assets = resp;
// Here render the first N.
for (node of resp.slice(0, LOAD_INITIAL_COUNT)) {
let li = renderAsset(node);
li.appendTo(ul_element);
}
// Don't bother with a 'load next' link if there is no more.
if (resp.length <= LOAD_INITIAL_COUNT) return;
// Construct the 'load next' link.
let load_next = $('<li>').addClass('load-next');
let link = $('<a>')
.attr('href', 'javascript:void(0);')
.click(function() { loadNext(ul_element); return false; })
.text('Load next')
.appendTo(load_next);
load_next.appendTo(ul_element);
});
});
};
}(jQuery));

View File

@@ -1,9 +0,0 @@
section.team
h2, .people-container
text-align: center
h3
margin-bottom: 0
h3 small
display: block
.people-intro, .row
margin-bottom: 20px

View File

@@ -1,502 +0,0 @@
.title-underline
padding-bottom: 5px
position: relative
margin-bottom: 20px
&:before
background-color: $primary
content: ' '
display: block
height: 2px
top: 125%
position: absolute
width: 50px
nav#nav-tabs,
nav#sub-nav-tabs
ul#nav-tabs__list,
ul#sub-nav-tabs__list
margin: 0
padding: 0
list-style: none
border-bottom: thin solid $color-background
+clearfix
li.nav-tabs__list-tab
float: left
border: none
border-bottom: 3px solid transparent
color: $color-text-dark-primary
user-select: none
&:hover
border-color: rgba($color-secondary, .3)
cursor: pointer
color: $color-text-dark
a
color: $color-text-dark
a
display: block
text-decoration: none
padding: 10px 15px 5px
color: $color-text-dark-primary
i
margin-right: 5px
color: $color-text-dark-secondary
font-size: .9em
&.pi-blender
margin-right: 10px
span
color: $color-text-dark-hint
margin-left: 5px
&.active
border-color: $color-secondary
color: $color-secondary-dark
a, i
color: $color-secondary-dark
&.disabled
border-color: $color-background-light
color: $color-text-dark-hint
cursor: default
a, i
color: $color-text-dark-hint
&:hover
border-color: $color-background-light
pointer-events: none
.dashboard-container
word-break: break-word
section.stream
ul.activity-stream__list
$activity-stream-thumbnail-size: 110px
> li
position: relative
display: flex
padding: 10px 0
overflow: hidden
border-top: thin solid $color-background-dark
&:first-child
border: none
&.active .activity-stream__list-details .title
color: $color-primary
&:hover
.title
text-decoration: underline
&.video
a.image
&:hover
i
font-size: 3.5em
img
opacity: .9
img
opacity: .7
z-index: 0
transition: opacity 150ms ease-in-out
i
+position-center-translate
z-index: 1
color: rgba(white, .6)
font-size: 3em
transition: font-size 100ms ease-in-out
&.comment
.activity-stream__list-details
padding: 0
.title
color: $color-text-dark
padding: 7px 10px 2px 10px
font-size: 1em
margin: 0
ul.meta
+list-meta
font-size: .9em
padding: 0 10px 7px 10px
li
&.where-parent:before
content: '\e83a'
font-family: 'pillar-font'
&.what:before
display: none
&.post
.activity-stream__list-thumbnail
border-color: $node-type-post
background-color: $node-type-post
.activity-stream__list-details .title
color: darken($node-type-post, 15%)
font:
size: 1.3em
weight: 500
&.asset, &.comment, &.post
&:hover
cursor: pointer
&.empty
display: none
color: $color-text-dark-primary
padding: 20px
text-align: center
span
color: $color-primary
&:hover
text-decoration: underline
cursor: pointer
&.with-picture
min-height: $activity-stream-thumbnail-size
.activity-stream__list-thumbnail
background-color: black
width: $activity-stream-thumbnail-size * 1.69
min-width: $activity-stream-thumbnail-size * 1.69
.activity-stream__list-thumbnail-icon
position: absolute
top: 0
left: 0
right: 0
bottom: 0
font-size: 1.3em
text-shadow: 1px 1px 0 rgba(black, .2)
background-image: linear-gradient(10deg, rgba(black, .5) 0%, transparent 40%)
i
position: absolute
bottom: -8px
left: 20px
top: initial
right: initial
color: white
.activity-stream__list-thumbnail
position: relative
display: flex
justify-content: center
align-items: center
overflow: hidden
width: 35px
height: auto
min-width: 35px
min-height: auto
+media-xs
display: none
&.image i
color: $node-type-asset_image
&.file i
color: $node-type-asset_file
&.video i
color: $node-type-asset_video
i
+position-center-translate
left: 23px
top: 21px
font-size: 1.1em
img
max-height: $activity-stream-thumbnail-size
+position-center-translate
.activity-stream__list-details
display: flex
flex-direction: column
justify-content: space-around
flex: 1
overflow: hidden
position: relative
max-width: 100%
padding: 10px 0
+media-xs
margin-left: 0
.ribbon
+ribbon
right: -47px
top: 5px
font-size: 12px
span
padding: 1px 50px
.title
padding: 0 10px
color: $color-text-dark
span
@include badge(hsl(hue($color-success), 60%, 45%), 3px)
font-size: .7em
padding: 1px 5px
margin-right: 5px
section.comments
padding: 0 15px 5px
ul
padding: 0
> ul
list-style-type: none
margin: 10px 0 0
> li
+text-overflow-ellipsis
border-top: thin solid $color-background-dark
padding: 10px 0
&:first-child
border: none
> a
+text-overflow-ellipsis
color: $color-text
display: block
padding-bottom: 5px
section.random-asset
border-bottom: thin solid $color-background-dark
ul.random-asset__list
list-style: none
padding: 0
> li
align-items: center
border-top: thin solid $color-background
display: flex
padding: 7px 0
position: relative
overflow: hidden
&:first-child
border-top: none
.ribbon
+ribbon
right: -47px
top: 5px
font:
size: 12px
weight: 500
z-index: 1
span
padding: 1px 50px
.random-asset__list-thumbnail
background-color: $color-background
display: block
height: 50px
margin-right: 15px
min-height: 50px
min-width: 50px
overflow: hidden
position: relative
width: 50px
img
width: 100%
i
+position-center-translate
font-size: 1.6em
color: $color-text-light
&.image
background-color: $node-type-asset_image
&.file
background-color: $node-type-asset_file
font-size: .8em
&.video
background-color: $node-type-asset_video
font-size: .8em
&.None
background-color: $node-type-group
.random-asset__list-details
.title
display: block
font-size: 1em
color: $color-text-dark-primary
&:hover
color: $color-primary
ul.meta
+list-meta
padding-top: 5px
font-size: .9em
li
&:before
left: -5px
&.what
text-transform: capitalize
&.featured
align-items: flex-start
flex-direction: column
padding: 0
a.title
font-size: 1.1em
padding: 10px 0 5px
display: block
color: $color-text
&:hover
color: $color-primary
a.random-asset__thumbnail
display: block
position: relative
&.video
background-color: black
img
opacity: .7
img
transition: opacity 150ms ease-in-out
width: 100%
max-width: 100%
i
+position-center-translate
color: white
font-size: 3em
text-shadow: 0 0 25px black
transition: font-size 150ms ease-in-out
&:hover
i
font-size: 3.5em
img
opacity: .85
ul.meta
+list-meta
padding-bottom: 10px
section.announcement
+container-box
margin-left: 15px
margin-right: 15px
.header-icons
display: flex
align-items: center
justify-content: center
padding: 20px 0 5px 0
i
font-size: 2.5em
color: $color-info
&.pi-heart-filled
color: $color-danger
margin-left: 5px
img.header
width: 100%
margin: 0 auto
border-top-left-radius: 3px
border-top-right-radius: 3px
iframe
width: 100%
position: relative
left: 15px
margin: 25px auto
+media-sm
height: 500px
+media-md
height: 520px
+media-lg
height: 580px
.text
padding: 15px
.title
padding-bottom: 10px
+media-xs
font-size: 1.4em
strong
color: $color-primary-dark
a
color: $color-text-dark-primary
.lead
font-size: 1em
+list-bullets
ul
margin-top: 10px
padding-left: 10px
hr
border: none
height: 1px
width: 100%
margin: 10px 0
background-color: $color-background
clear: both
+media-xs
padding-left: 10px
.buttons
margin: 15px auto 0 auto
display: flex
align-items: center
justify-content: space-around
flex-wrap: wrap
body.homepage
.blog
// Custom tweak to Bootstrap grid for the only case when
// the post is inside a column (it's usually centered in the page).
.col-md-9
flex: 1
max-width: 100%
.jumbotron
padding-top: 6em
padding-bottom: 6em
*
font-size: $h1-font-size
.lead
font-size: $font-size-base

View File

@@ -1,39 +0,0 @@
.services
#page-header
text-shadow: 1px 1px 0 rgba(black, .2)
border-bottom: none
align-items: initial
.page-title
text-align: left
font-size: 3em
margin: 0
.page-title-summary
max-width: 900px
text-align: left
font-size: 1.4em
.page-card-side
img
max-width: 100%
border-radius: 3px
.tip
margin-top: 15px
color: $color-text-dark-secondary
font-size: .8em
a
color: $color-primary
text-decoration: underline
span.text-background
+text-background(white, #358, 2px, 5px 0)
.navbar-backdrop-overlay
background-image: linear-gradient(rgba(black, .3), rgba(black, .8))
#blender-sync
small strong
color: $color-success

View File

@@ -1,380 +0,0 @@
.join
position: relative
nav.navbar
background-color: white
.navbar-brand
color: $color-text
li a.navbar-item
color: $color-text
.navbar-toggle
border: 2px solid $color-text-dark-primary
color: $color-text
.navbar-nav
+media-xs
padding: 10px
#page-header
+media-lg
min-height: 600px
+media-xs
background-size: cover
background-position: right
min-height: 500px
.page-title,
.page-title-summary
line-height: 1.2em
text-align: left
text-shadow: 1px 1px 0 rgba(black, .5), 0 0 25px rgba(black, .5)
.page-title-summary
font-size: 1.4em
li.special
color: $color-success
font-weight: bold
.page-card
&:nth-child(even)
background-color: white
&-header
margin-bottom: 0
&-side
+media-xs
width: 100%
max-width: 100%
&.text
+media-xs
padding-left: 50px
padding:
bottom: 150px
top: 150px
&.right
background-color: $color-background-light
.page-card-title,
.page-card-summary
padding-left: 25px
.page-card-title:after
left: 25px
.text
+media-xs
padding-left: 25px
&.intro
+media-xs
margin-left: 50px
.page-card-side
+media-sm
max-width: 500px
max-width: 600px
padding:
bottom: 0
top: 50px
&+.page-card-header
padding-top: 0
.page-card-summary
font-size: 1.2em
.page-card-image
align-items: center
display: flex
height: 100%
justify-content: center
&.light
overflow: hidden
position: relative
.learn
color: white
text-decoration: underline
.page-card
&-title
color: white
font-weight: bold
&:after
border-color: white
&-summary
color: white
position: relative
text-shadow: 1px 1px 0 rgba(black, .5), 0 0 25px rgba(black, .2)
z-index: 1
p
font-size: .9em
&.summary-action
font-size: .8em
a.page-card-cta
font-size: .9em
a
color: white
text-decoration: underline
a.page-card-cta
background: #ff4970
border-radius: 3px
border: none
box-shadow: 1px 1px 0 rgba(black, .2)
font-weight: bold
margin-right: 15px
margin-top: 25px
padding: 7px 20px
text-decoration: none
text-shadow: none
&:hover
background: lighten(#ff4970, 5%)
.page-card-image
img
+media-xs
display: none
max-width: initial
position: absolute
width: initial
z-index: 0
&.training
margin-top: 45px
a.page-card-cta
background: #0082ff
.page-card-image
img
right: 50px
top: 50%
transform: translateY(-50%)
&.open-movies
.page-card-image
img
top: 50px
left: 50px
&.services
.page-card-image
img
left: 50px
&.subscribe
padding: 40px 0
text-align: center
span strong
color: $color-danger
.page-card-side
width: 100%
max-width: 100%
.page-card-title
line-height: 1.5em
color: white
text-align: center
padding-bottom: 15px
&:after
border: none
.page-card-summary
color: $color-text-light-primary
.page-card-cta
text-align: center
font-size: 1.3em
padding: 7px 35px
margin-right: initial
border: thin solid rgba(white, .8)
border-radius: 3px
.training-other
color: $color-text-dark-secondary
font-size: .9em
padding: 30px
text-align: center
a
color: $color-text-dark-secondary
text-decoration: underline
&:hover
color: $color-text-dark-primary
.pricing
+media-xs
padding-top: 30px
margin-bottom: 0
padding-bottom: 100px
.supported-by
padding: 50px 0
text-align: center
background-color: $color-background
border-top: 5px solid $color-background-dark
img.logos
padding: 60px 0 100px 0
max-height: 500px
max-width: 100%
+media-xs
max-width: 100%
.assets
padding-bottom: 80px
.flex
+media-xs
flex-direction: column
display: flex
.box
flex: 1
margin: 20px
.page-triplet-container
.triplet-card
+media-xs
margin-top: 20px
.container.wide-on-sm
+media-sm
width: 100%
section.pricing
padding: 50px 0
margin-bottom: 25px
background-color: $color-background-light
+clearfix
+media-xs
padding: 0
h2
text-align: center
font-size: 2.3em
text-shadow: 1px 1px 0 white
margin:
top: 10px
bottom: 50px
padding: 0
+media-xs
font-size: 1.6em
.box
margin-top: 40px
padding: 20px 20px 60px 20px
border: 1px solid $color-text-dark-hint
background-color: white
position: relative
text-align: center
border-top: 3px solid rgba($color-text-dark-secondary, .5)
border-bottom-left-radius: 3px
border-bottom-right-radius: 3px
+media-xs
margin-bottom: 15px
&.yearly
border-top: 3px solid $color-info
transform: scale(1.1)
+media-xs
transform: scale(1)
a.sign-up-now
+button($color-primary, 3px, true)
h3
font-size: 1.8em
padding-bottom: 0
margin: 25px 0 0 10px
.pricing-display
position: relative
color: $color-info
padding: 10px 0
.currency-sign,
.digit-dec
font-size: 1.7em
position: relative
top: -25px
font-weight: 100
.digit-int
font-size: 4.5em
font-weight: 100
.pricing-caption
color: $color-text-dark-hint
text-align: center
padding-bottom: 25px
ul
text-align: left
+list-bullets
ul
color: $color-text-dark-primary
padding-left: 10px
margin-top: 15px
li
padding-bottom: 8px
.sign-up-now
position: absolute
bottom: 25px
width: 65%
left: 50%
transform: translateX(-50%)
font-size: 1.2em
+button($color-primary, 3px)
padding: 5px 25px
white-space: nowrap
text-align: center
.education
color: $color-text-dark-primary
font-size: 1.1em
padding: 25px 15px 0
text-align: center
h3
color: $color-primary-dark
.btn
margin-top: 15px
min-width: 200px

View File

@@ -1,99 +0,0 @@
// Bootstrap variables and utilities.
@import "../../node_modules/bootstrap/scss/functions"
@import "../../node_modules/bootstrap/scss/variables"
@import "../../node_modules/bootstrap/scss/mixins"
// Pillar variables and utilities.
@import "../../../pillar/src/styles/config"
@import "../../../pillar/src/styles/utils"
// Bootstrap components.
@import "../../node_modules/bootstrap/scss/root"
@import "../../node_modules/bootstrap/scss/reboot"
@import "../../node_modules/bootstrap/scss/type"
@import "../../node_modules/bootstrap/scss/images"
@import "../../node_modules/bootstrap/scss/code"
@import "../../node_modules/bootstrap/scss/grid"
@import "../../node_modules/bootstrap/scss/tables"
@import "../../node_modules/bootstrap/scss/forms"
@import "../../node_modules/bootstrap/scss/buttons"
@import "../../node_modules/bootstrap/scss/transitions"
@import "../../node_modules/bootstrap/scss/dropdown"
@import "../../node_modules/bootstrap/scss/button-group"
@import "../../node_modules/bootstrap/scss/input-group"
@import "../../node_modules/bootstrap/scss/custom-forms"
@import "../../node_modules/bootstrap/scss/nav"
@import "../../node_modules/bootstrap/scss/navbar"
@import "../../node_modules/bootstrap/scss/card"
@import "../../node_modules/bootstrap/scss/breadcrumb"
@import "../../node_modules/bootstrap/scss/pagination"
@import "../../node_modules/bootstrap/scss/badge"
@import "../../node_modules/bootstrap/scss/jumbotron"
@import "../../node_modules/bootstrap/scss/alert"
@import "../../node_modules/bootstrap/scss/progress"
@import "../../node_modules/bootstrap/scss/media"
@import "../../node_modules/bootstrap/scss/list-group"
@import "../../node_modules/bootstrap/scss/close"
@import "../../node_modules/bootstrap/scss/modal"
@import "../../node_modules/bootstrap/scss/tooltip"
@import "../../node_modules/bootstrap/scss/popover"
@import "../../node_modules/bootstrap/scss/carousel"
@import "../../node_modules/bootstrap/scss/utilities"
@import "../../node_modules/bootstrap/scss/print"
// Pillar components.
@import "../../../pillar/src/styles/apps_base"
@import "../../../pillar/src/styles/error"
@import "../../../pillar/src/styles/components/base"
@import "../../../pillar/src/styles/components/jumbotron"
@import "../../../pillar/src/styles/components/alerts"
@import "../../../pillar/src/styles/components/navbar"
@import "../../../pillar/src/styles/components/dropdown"
@import "../../../pillar/src/styles/components/footer"
@import "../../../pillar/src/styles/components/shortcode"
@import "../../../pillar/src/styles/components/statusbar"
@import "../../../pillar/src/styles/components/search"
@import "../../../pillar/src/styles/components/flyout"
@import "../../../pillar/src/styles/components/forms"
@import "../../../pillar/src/styles/components/inputs"
@import "../../../pillar/src/styles/components/buttons"
@import "../../../pillar/src/styles/components/popover"
@import "../../../pillar/src/styles/components/tooltip"
@import "../../../pillar/src/styles/components/checkbox"
@import "../../../pillar/src/styles/components/overlay"
@import "../../../pillar/src/styles/components/card"
@import "../../../pillar/src/styles/comments"
@import "../../../pillar/src/styles/notifications"
/* Blender Cloud specific styles */
@import "../../../pillar/src/styles/_project"
@import "../../../pillar/src/styles/_project-sharing"
@import "../../../pillar/src/styles/_project-dashboard"
@import "../../../pillar/src/styles/_user"
@import _welcome
@import _homepage
@import _services
@import _about
@import "../../../pillar/src/styles/_search"
@import "../../../pillar/src/styles/_organizations"
/* services, about, etc */
@import "../../../pillar/src/styles/_pages"
/* plugins are included here, don't include in base unless needed by other pillar apps */
@import "../../../pillar/src/styles/plugins/_jstree"
@import "../../../pillar/src/styles/plugins/_js_select2"
/* CSS for pillar-font comes from fontello.com using static/assets/font/config.json */

Some files were not shown because too many files have changed in this diff Show More