[Django + micro channel small developers] 08Django layer optimization model

Django model layer optimization

First, my ultimate goal is to develop PythonDjango + campus micro-channel application applets, so I see this series you can view some useful elements, but some elements are the notes I need to record the record, not a collection of the entire project.

Django model layer change and migration:

• makemigration appname

• migrate to execute database

• selmigrate appname 0014 to see the implementation of sql statement (20)

• showmigration appname change history display applications and migration history

Django Model layer - lazy loading, preloaded

• The root of the problem lazy loading

  • Opaque ORM framework

    • After a query ORM framework would turn into two SQL statements

    • Two stitches it through ORM framework can not be turned into a query

  • Lazy loading

    • There is a foreign key relationships and many to many

    • Does not retrieve data associated object

    • Call the associated object will query the database again

  • View DjangoORM data loading

    • Indeed two requests

• Pre-loaded method

  • Single Association Object preload -select_related

  • Preloading the plurality of objects associated -prefetch_related

  • user = User.object.prefetch_related(“menu”)

  • user[0].menu.all()

• Comparative Performance Data

  • Preloading load much faster than the lazy

  • To a query object associated objects, it must be pre-loaded (to improve performance)

Django layer optimization model (long link)

• Processing logic database connection

  • Each request will be repeatedly connected database

  • Each repetition often: -1--, --1--, often a connection: -1111-

  • Handle high concurrent requests to the service also to the enormous pressure

  • Unable to carry higher concurrent service

• Avoid negative optimization

  • Position storage database connection: thread-local variables

  • The maximum number of connections supported by the database

    • Open a command line navicat
    • show variables like “%max_connection%”;
    • set global max_connections = 200 to 200

  • CONN_MAX_AGE (maximum number of connections) Configuration Guide

    • Number of deployment threads: n <database maximum number of connections
    • Development model is not recommended for use CONN_MAX_AGE
    • Each request will create a thread

• Theoretical results

  • Establish or close events (5ms) DB connections

  • DB query event 15ms

  • Short link utilization: 15 / (15 + 5 * 2) = 0.6

  • Long link utilization 15 3 (15 3 + 5 * 2) = 0.82

Django database model layer Practices

• The proper use of the index

  • Index should be indexed column
  • Not the index should not be indexed column

• Use iterator iterator iteration QuerySet

  • When QuerySet very large, iterators save memory

• Understand property cache object

  • Can not invoke the property: it is the default data does not contain the foreign key
  • Callable properties: it is a foreign key, data, and many to many, may be a function call,
  • The callable Python attributes assigned to a variable, when called again, you do not need to call the database, but you can call the data Python variable (to improve performance)

• Work on the database to make the database

  • Filter: exclude, filter attributes
  • Polymerization: polymerization using function annotate
  • When necessary, use native SQL

• The right to retrieve data

  • Field being indexed retrieval using
  • Using the unique modified Field Search

• Do not make unnecessary retrieval

  • Use QuerySet Values ​​(), Value_list () function returns a structure of the container Python
  • The length of the query results using QuerySet.count () instead of len (QuerySet)
  • Whether a judge is empty yo QuerySet.exists () instead if QuerySet

• Do not make unnecessary sorting

  • Do not make unnecessary sorting

• Batch operations

  • Large amounts of data, bulk operations