Logging in Python for large applications

Why use logging Basics When to use what? Logbook Logging in Python For complex applications Fayaz Yusuf Khan 20 December 2013 Error reporting The end

Why use logging Basics When to use what? Logbook Why use logging? Error reporting The end

Why use logging Basics When to use what? Logbook Why use logging? • All errors happen in runtime. Error reporting The end

Why use logging Basics When to use what? Logbook Why use logging? • All errors happen in runtime. • Testing can never be 100% Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting import logging logging . warning ( ’ Watch out ! ’) # will print a message logging . info ( ’I told you so ’) # will not print a n y t h i n g The end

Why use logging Basics When to use what? Logbook Error reporting import logging logging . warning ( ’ Watch out ! ’) # will print a message logging . info ( ’I told you so ’) # will not print a n y t h i n g Output: WARNING:root:Watch out! The end

Why use logging Basics When to use what? Logbook Error reporting Logging to a ﬁle import logging logging . basicConfig ( filename = ’ example . log ’ , level = logging . DEBUG ) logging . debug ( ’ This message should go to the log file ’) logging . info ( ’ So should this ’) logging . warning ( ’ And this , too ’) The end

Why use logging Basics When to use what? Logbook Error reporting Logging to a ﬁle import logging logging . basicConfig ( filename = ’ example . log ’ , level = logging . DEBUG ) logging . debug ( ’ This message should go to the log file ’) logging . info ( ’ So should this ’) logging . warning ( ’ And this , too ’) Output: DEBUG:root:This message should go to the log file INFO:root:So should this WARNING:root:And this, too The end

Why use logging Basics When to use what? Logbook Initializing logger = logging . getLogger ( __name__ ) Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting The end print() Display console output for ordinary usage of a command line script or program

Why use logging Basics When to use what? Logbook Error reporting logging.info() Report events that occur during normal operation of a program (e.g. for status monitoring or fault investigation) The end

Why use logging Basics When to use what? Logbook Error reporting logging.debug() Very detailed output for diagnostic purposes The end

Why use logging Basics When to use what? Logbook Error reporting warnings.warn() In library code if the issue is avoidable and the client application should be modiﬁed to eliminate the warning. The end

Why use logging Basics When to use what? Logbook Error reporting logging.warning() If there is nothing the client application can do about the situation, but the event should still be noted The end

Why use logging Basics When to use what? Logbook Error reporting Report an error regarding a particular runtime event The end

Why use logging Basics When to use what? Logbook Error reporting Raise an exception Report an error regarding a particular runtime event The end

Why use logging Basics When to use what? Logbook Error reporting logging.error() or logging.exception() Report suppression of an error without raising an exception (e.g. error handler in a long-running server process) The end

Why use logging Basics When to use what? Logbook Error reporting The end logging.critical() A serious error, indicating that the program itself may be unable to continue running.

Why use logging Basics When to use what? Logbook Error reporting Logbook pythonhosted.org/Logbook Logbook is a logging sytem for Python that replaces the standard library’s logging module. The end

Why use logging Basics When to use what? Logbook Error reporting >>> from logbook import Logger >>> log = Logger ( ’ Logbook ’) >>> log . info ( ’ Hello , World ! ’) [2010 -07 -23 16:34] INFO : Logbook : Hello , World ! The end

Why use logging Basics When to use what? Logbook Error reporting Context managers everywhere! from logbook import SyslogHandler error_handler = SyslogHandler ( ’ logbook example ’ , level = ’ ERROR ’) with error_handler . a p p l i c a t i o n b o u n d (): # wh a t e v e r is e x e c u t e d here and an error is logged to # the error handler ... The end

Why use logging Basics When to use what? Logbook Error reporting Getting your libraries and old code to behave from logbook . compat import r e d i r ec t _ l o g g i n g r e d i r e ct _ l o g g i n g () The end

Why use logging Basics When to use what? Logbook Error reporting Injecting context import os from logbook import Processor def inject_cwd ( record ): record . extra [ ’ cwd ’] = os . getcwd () with my_handler . a p p l i c at i o n b o u n d (): with Processor ( inject_cwd ). a p p l i c a t i o n b o u n d (): # e v e r y t h i n g logged here will have the current # working d i r e c t o r y in the log record . ... The end

Why use logging Basics When to use what? Logbook Error reporting Formatting in Logging import logging logging . basicConfig ( format = ’ %( levelname ) s :%( message ) s ’ , level = logging . DEBUG ) The end

Why use logging Basics When to use what? Logbook Error reporting Formatting in Logbook >>> from logbook import StderrHandler >>> handler = StderrHandler ( ... format_string = ’{ record . channel }: { record . message } ’) >>> handler . formatter < logbook . handlers . St ri n gF or ma t te r object at 0 x100641b90 > The end

Why use logging Basics When to use what? Logbook Custom formatters >>> def my_formatter ( record , handler ): ... return record . message ... >>> handler . formatter = my_formatter Error reporting The end

Why use logging Basics When to use what? Logbook Custom formatters >>> def my_formatter ( record , handler ): ... return record . message ... >>> handler . formatter = my_formatter Tip: Or use Jinja! Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting The end Extra ﬁelds handler = StderrHandler ( format_string = ’{ record . channel }: { record . message ) [{ record . extra [ cwd ]}] ’)

Why use logging Basics When to use what? Logbook Redis import logbook from logbook . queues import RedisHandler handler = RedisHandler () l = logbook . Logger () with handler : l . info ( ’ Your log message ’) Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting RabbitMQ Handler handler = R a bb it MQ H an dl er ( ’ amqp :// guest : gu e st @l oc a lh os t // ’ , queue = ’ my_log ’) Subscriber with target_h andler : subscriber = R a b b i t M Q S u b s c r i b e r ( ’ amqp :// guest : gu e st @l oc a lh os t // ’ , queue = ’ my_log ’) subscriber . d i s p a t ch _ f o r e v e r () The end

Why use logging Basics When to use what? Logbook ZeroMQ Don’t use it. (It’s broken) Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting The end Minimize logging >>> from logbook import FingersCrossedHandler , StderrHandler >>> handler = F i n g e r s C r o s s e d H a n d l e r ( StderrHandler ()) >>> with handler . a p p l i c a t i o n b o u n d (): ... logbook . info ( ’ Hello ’) ... ... >>> handler = F i n g e r s C r o s s e d H a n d l e r ( StderrHandler ()) >>> with handler . a p p l i c a t i o n b o u n d (): ... logbook . info ( ’ Hello ’) ... logbook . error ( ’ Bye ’) ... ... [2013 -12 -16 11:12] INFO : Generic : Hello [2013 -12 -16 11:12] ERROR : Generic : Bye

Why use logging Basics When to use what? Logbook Error reporting TestHandler >>> from logbook import TestHandler , Logger >>> logger = Logger ( ’ Testing ’) >>> handler = TestHandler () >>> handler . push_thread () >>> logger . warn ( ’ Hello World ’) >>> handler . records [ < logbook . base . LogRecord object at 0 x100640cd0 >] >>> handler . f o r m a t t e d _ r e c o r d s [ u ’[ WARNING ] Testing : Hello World ’] The end

Why use logging Basics When to use what? Logbook Github github.com/mitsuhiko/logbook Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting Emails? Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting The end Airbrake Airbrake is the leading exception reporting service, currently providing error tracking for 50,000 applications with support for 18 programming languages.

Why use logging Basics When to use what? Logbook Sign up airbrake.io Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting Flask from flask import Flask , request from airbrake import A i r b r a k e E r r o r H a n d l e r import gevent @app . errorhandler (500) def interna l_error ( error ): if app . config [ ’ E X C E P T I O N _ L O G G I N G ’ ]: handler = A i r b r a k e E r r o r H a n d l e r ( api_key = app . config [ ’ A I R B R E A K _ A P I _ K E Y ’] , env_name = ENV , request_url = request . url , request_path = request . path , requ est_meth od = request . method , request_args = request . args , r eq ue st _ he ad er s = request . headers ) gevent . spawn ( handler . emit , error ) The end

Why use logging Basics When to use what? Logbook Error reporting Django AIRBRAKE = { ’ API_KEY ’: ’ y o u r _ a p i _ k e y _ h e r e ’ , ’ USE_SSL ’: True , ’ TIMEOUT ’: 5 , ’ ENVIRONMENT ’: ’ u n i q u e _ n a m e _ f o r _ e n v i r o n m e n t ’ , } The end

Why use logging Basics When to use what? Logbook Errbit github.com/errbit/errbit Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting The end Sentry getsentry.com Sentry is a realtime event logging and aggregation platform. At its core it specializes in monitoring errors and extracting all the information needed to do a proper post-mortem without any of the hassle of the standard user feedback loop.

Why use logging Basics When to use what? Detail Logbook Error reporting The end

Why use logging Basics When to use what? Logbook More detail Error reporting The end

Why use logging Basics When to use what? Logbook Install the server sentry.readthedocs.org Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting Sentry + Logbook raven.readthedocs.org/en/latest/conﬁg/logbook.html Install raven pip install raven Your app code will have this at the entry point from raven . handlers . logbook import SentryHandler client = Client (...) sent ry_handl er = SentryHandler ( client ) with sentry_h andler . a p pl i c a t i o n b o u n d (): # e v e r y t h i n g logged here will go to sentry . ... The end

Why use logging Basics When to use what? Logbook Error reporting Pay attention to formatting! Sentry will intelligently group messages if you use proper string formatting. logger . error ( ’ There was some {} error ’ , ’ crazy ’) logger . error ( ’ There was some {} error ’ , ’ fun ’) logger . error ( ’ There was some {} error ’ , 1) The end

Why use logging Basics When to use what? Logbook Tagging logger . error ( ’ Could not do that ’ , tags ={ ’ user email ’: ’ foo@bar ’ }) Error reporting The end

Why use logging Basics When to use what? Logbook Error reporting The end TicketingHandler Create unique tickets for log records and keep track of the number of times these log records were created from logbook import ERROR from logbook . ticketing import T i c k e t i n g H a n d l e r handler = T i c k e t i n g Ha n d l e r ( ’ postgres :// localhost / database ’ , level = ERROR ) with handler : # e v e r y t h i n g in this block and thread will be handled by # the t i c k e t i n g d a t a b a s e handler ...

Why use logging Basics When to use what? Logbook Error reporting Phew! It’s ﬁnally over! Reach me at fayaz at dexetra dot com. The end

Logging in Python for large applications

More Related Content

What's hot

Viewers also liked

Similar to Logging in Python for large applications

Recently uploaded

Logging in Python for large applications