ProPeler
Home
Console
Upload
information
Create File
Create Folder
About
Tools
:
/
opt
/
cpanel-ccs
/
txdav
/
who
/
Filename :
cache.pyc
back
Copy
� �t�ac @ s� d Z d g Z d d l Z d d l Z d d l Z d d l m Z d d l m Z m Z d d l m Z d d l m Z m Z d d l m Z d d l m Z d d l m Z m Z m Z m Z d d l m Z d d l m Z d d l m Z d d l! m" Z" d d l# m Z m Z d d l$ m% Z% m& Z& m' Z' m( Z( e � Z) d e% f d � � YZ* d e f d � � YZ+ d e, f d � � YZ- e e e � d e e" f d � � Y� Z. d S( s Caching Directory Service t CachingDirectoryServicei����N( t implementer( t ClientFactoryt MemcacheError( t config( t inlineCallbackst returnValue( t Logger( t DirectoryService( t IDirectoryServicet FieldNamet RecordTypet DirectoryServiceError( t ConstantsContainer( t IStoreDirectoryService( t CalendarDirectoryServiceMixin( R R ( t Valuest ValueConstantt NamedConstantt Namest IndexTypec B s> e Z d Z e d � Z e d � Z e d � Z e d � Z RS( s2 Constants to use for identifying indexes t uidt guidt shortNamet emailAddress( t __name__t __module__t __doc__R R R R R ( ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR 8 s t DirectoryMemcacheErrorc B s e Z d Z RS( s- Error communicating with memcached. ( R R R ( ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR B s t DirectoryMemcacherc B sq e Z d Z d Z d � Z e d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z RS( s� Provide a cache of directory records in memcached so that worker processes and the DPS processes across multiple app servers can share a cache and thus reduce load on the directory server. i c C s5 | | _ | | _ | | _ d t j | f | _ d S( Ns %d%s( t _cacheTimeoutt _recordServicet _realmNameR t KEY_VERSIONt _keyVersion( t selft cacheTimeoutt recordServicet realmNamet keyModifier( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt __init__Q s c C s� | s t | d � r� t j j j j rF d j t j j j j � } n* d j t j j j j t j j j j � } t j | g d d d d �| _ n | j S( s� Get the memcache client instance to use for caching. @param refresh: whether or not to create a new memcache client @type refresh: L{bool} @return: the client to use @rtype: L{memcacheclient.Client} t memcacheClients unix:{}s {}:{}t debugi t pickleProtocoli ( t hasattrR t Memcachedt Poolst Defaultt MemcacheSockett formatt BindAddresst PortR t getClientR) ( R# t refresht client_addr( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt _getMemcacheClientW s $c C s� i } x� | j j � D]� \ } } | j j j | � } | t t f k rY | | | j <q | t j k r~ t | � | | j <q t | t t f � r | r� | j n d | | j <q q W| j | f S( N( t fieldst iteritemst servicet fieldNamet valueTypet unicodet boolt namet uuidt UUIDt strt issubclassR R t Nonet __class__( R# t recordR8 t fieldt valueR< ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt pickleRecordm s #c C s� | \ } } t | j t � r. | j j | � Si } xt| j � D]f\ } } y | j j j | � } Wn t k ry qA X| j j j | � } | t t f k r� | | | <qA | t j k r� t j | � | | <qA t | t � r| d k r| j j | � | | <q�d | | <qA t | t � rA | d k r�y t j | � | | <Wq�t k r�y t j | � | | <Wq�t k r�t j d j | � � t � q�Xq�Xq�qA qA W| | j | � Sd S( Nt recordTypes Failed to lookup record type: {}( t isinstanceR t DPSClientDirectoryServicet _dictToRecordR9 R; t lookupByNamet ValueErrorR< R= R> R@ RA RC R RD R t BaseRecordTypeR t logt errorR1 R ( R# t datat record_classt record_fieldsR8 R; RH RG R< ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt unpickleRecordy s: c C s | j | | j | � � d S( s Store a record in memcache. @param key: memcache key to use @type key: L{str} @param record: record to store @type record: L{DirectoryRecord} @raise: L{DirectoryMemcacheError} if failure to store in memcache N( t memcacheSetRI ( R# t keyRF ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt memcacheSetRecord� s c C s� t j | � } | j � j | | d | j �s� t j d � | j d t � j | | d | j �s� t j d � | ` t d � � q� n d S( s� Store a value in memcache. @param key: memcache key to use @type key: L{str} @param value: value to store @type value: L{str} @raise: L{DirectoryMemcacheError} if failure to store in memcache t times% Could not write to memcache, retryingR5 s, Could not write to memcache again, giving ups Failed to write to memcacheN( t base64t b64encodeR7 t setR RQ RR t TrueR) R ( R# RX RH ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyRW � s ! c C s, | j | � } | d k r( | j | � Sd S( s- Try to get a record from memcache. @param key: the memcache key to use @type key: L{str} @return: any directory record found or L{None} @rtype: L{DirectoryRecord} or L{None} @raise: L{DirectoryMemcacheError} if failure to read from memcache N( t memcacheGetRD RV ( R# RX t pickled( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt memcacheGetRecord� s c C s� t j | � } y | j � j | � } Wnp t k r� t j d � y | j d t � j | � } Wq� t k r� t j d � | ` t d � � q� Xn X| S( s Try to get a record from memcache. @param key: the memcache key to use @type key: L{str} @return: any value found or L{None} @rtype: L{str} or L{None} @raise: L{DirectoryMemcacheError} if failure to read from memcache s& Could not read from memcache, retryingR5 s- Could not read from memcache again, giving ups Failed to read from memcache( R[ R\ R7 t getR RQ RR R^ R) R ( R# RX RH ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR_ � s c C sX t | t � r7 d | j | j | j | d | d f Sd | j | j | j | f Sd S( s� Return a key that can be used to store/retrieve a record in memcache. if short-name is the indexType the recordType be encoded into the key. @param indexType: one of the IndexType values @type indexType: L{str} @param indexKey: the value being indexed @type indexKey: L{str} @return: a memcache key comprised of the passed-in values and the directory service's realm @rtype: L{str} s dir|v%s|%s|%s|%s|%si i s dir|v%s|%s|%s|%sN( RK t tupleR" R RH ( R# t indexTypet indexKey( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt generateMemcacheKey� s c C s | j � j � d S( s� Flush all records from memcache. Note this is only for testing and must not be called in a production setup because it flushes everything from memcache N( R7 t flush_all( R# ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt flush s ( R R R R! R( t FalseR7 RI RV RY RW Ra R_ Rf Rh ( ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR H s ' c B s� e Z d Z e e e f � Z d d e d � Z d � Z d � Z d � Z d � Z e d � Z d � Z d � Z d � Z d � Z e d d � � Z e d d � � Z e d d � � Z e d d d � � Z e d � � Z d � Z d d d d d � Z d d d � Z e d � Z d � Z d d d � Z d d d d � Z! d d d � Z" d � Z# d d � Z$ d � Z% e d � � Z& d � Z' RS( s� Caching directory service. This is a directory service that wraps an L{IDirectoryService} and caches directory records. i i c C s� t j | | j � | | _ | j | _ | j | _ | j | _ | j | _ | j | _ | j | _ | j | _ | j | _ | | _ | d k r� t | _ n t | _ | | _ | | _ | j � d S( Ni ( t BaseDirectoryServiceR( R& t _directoryt recordWithUIDt _wrapped_recordWithUIDt recordWithGUIDt _wrapped_recordWithGUIDt recordWithShortNamet _wrapped_recordWithShortNamet recordsWithEmailAddresst _wrapped_recordsWithEmailAddresst _expireSecondsRi t _purgingEnabledR^ t _lookupsBetweenPurgest negativeCachingt resetCache( R# t directoryt expireSecondst lookupsBetweenPurgesRw ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR( % s" c C s | | _ d S( s9 Replace the default no-op timing method N( t _addTiming( R# t f( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt setTimingMethodD s c C s d S( s� Timing won't get recorded by default -- you must call setTimingMethod with a callable that takes a key such as a method name, and a duration. N( ( R# RX t duration( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR| J s c C s t j d � i i t j 6i t j 6i t j 6i t j 6| _ i i t j 6i t j 6i t j 6i t j 6| _ d | _ d | _ | j r� | j | _ n t j j j j rt j j s� t | j t � rt | j | j | j j t j j r� d n d � | _ n d | _ d S( s! Clear the cache s Resetting cachei t at bN( RQ R* R R R R R t _cachet _negativeCachet _hitCountt _requestCountRu Rv t _lookupsUntilScanR R- R. R/ t ClientEnabledt DirectoryProxyt EnabledRK Rk RL R Rt R& t _memcacherRD ( R# ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyRx Q s. !c C s | | _ d S( s� Only used for unit tests to override the notion of "now" @param timestamp: seconds @type timestamp: C{float} N( t _test_time( R# t timestamp( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt setTestTimev s c C sd t | d � r | j } n t j � } g } t j | k ru | | f | j t j | j <| j t j | j f � n t j | k r� y: | | f | j t j | j <| j t j | j f � Wq� t k r� q� Xn t j | k r[y` | j j } xM | j D]B } | | f | j t j | | f <| j t j | | f f � q� WWq[t k rWq[Xn t j | k r�yH xA | j D]6 } | | f | j t j | <| j t j | f � qwWWq�t k r�q�Xn | r`| j d k r`x | D]t \ } } | j j | | � } t j d | f � y | j j | | � Wq�t k rXt j d | f � q�Xq�Wn d S( s� Store a record in the cache, within the specified indexes @param record: the directory record @param indexTypes: an iterable of L{IndexType} R� s Memcache: storing %ss Memcache: failed to store %sN( R, R� RZ R R R� t appendR t AttributeErrorR RJ R? t shortNamesR t emailAddressesR� RD Rf RQ R* RY R RR ( R# RF t indexTypest addToMemcacheR� t cachedt typeNameR? R Rd RX t memcachekey( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt cacheRecord sJ $ c C s� t | d � r | j } n t j � } | | j | | <| j d k r� | j j | | � } y | j j d | f | � Wq� t k r� t j d | f � q� Xn t j d d | j d | �d S( s� Store a record in the negative cache, within the specified indexes @param record: the directory record @param indexType: an L{IndexType} R� s -%ss Memcache: failed to store -%ss' Directory negative cache: {index} {key}t indexRX N( R, R� RZ R� R� RD Rf RW R RQ RR R* RH ( R# Rd RX R� R� ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt negativeCacheRecord� s c C s@ | j | j t j k r0 | j t j | j =n y4 | j | j t j k rc | j t j | j =n Wn t k rw n XyZ | j j } xG | j D]< } | | f } | | j t j k r� | j t j | =q� q� WWn t k r� n XyB x; | j D]0 } | | j t j k r� | j t j | =q� q� WWn t k r;n Xd S( sl Remove a record from all indices in the cache @param record: the directory record N( R R� R R R� RJ R? R� R R� R ( R# RF R� R? RX R ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt purgeRecord� s* c C s� t | d � r | j } n t j � } x_ | j D]T } xK | j | j � D]6 \ } \ } } | | j | k rK | j | | =qK qK Wq1 Wd S( sF Scans the cache for expired records and deletes them R� N( R, R� RZ R� t itemsRt ( R# t nowRd RX t cachedTimet _ignore_record( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt purgeExpiredRecords� s &c C s� | j rC | j d k r1 | j | _ | j � qC | j d 8_ n t | d � r^ | j } n t j � } | j d 7_ | | j | k rT| j | j | d � \ } } | | j | k rt j d d | j d | �| j | � | j d j | � d � qTt j d d | j d | �| j d 7_ | j d j | � d � | t f Sn | j r�yu | j | | } | | | j k r�t j d d | j d | �| j d j | � d � d t f S| j | | =Wq�t k r�q�Xn | j d k rc| j j | | � } t j d | f � y | j j | � } Wn+ t k rdt j d | f � d } n X| d k r�t j d | f � nF t j d | f � | j | t j t j t j f d t �| t f S| j rcy | j j! d | f � } Wn+ t k r!t j d | f � d } n X| d k r`t j d | f � | | j | | <d t f Sqcn t j d d | j d | �| j d j | � d � d t" f S( so Looks for a record in the specified index, under the specified key. After every config.DirectoryCaching.LookupsBetweenPurges lookups are done, purgeExpiredRecords() is called. @param index: an index type @type indexType: L{IndexType} @param key: the key to look up in the specified index @type key: any valid type that can be used as a dictionary key @return: tuple of (the cached L{DirectoryRecord}, or L{None}) and a L{bool} indicating whether a query will be required (not required if a negative cache hit) @rtype: L{tuple} i i R� g s- Directory cache miss (expired): {index} {key}R� RX s {}-expireds" Directory cache hit: {index} {key}s {}-hits+ Directory negative cache hit: {index} {key}s {}-neg-hits Memcache: checking %ss Memcache: failed to get %ss Memcache: miss %ss Memcache: hit %sR� s -%ss Memcache: failed to get -%ss Memcache: negative hit %ss# Directory cache miss: {index} {key}s {}-missN( g N(# Ru R� Rv R� R, R� RZ R� R� Rb RD Rt RQ R* RH R� R| R1 R� Ri Rw R� t KeyErrorR� Rf Ra R RR R� R R R R R_ R^ ( R# Rd RX R? R� R� RF t disabledTimeR� t val( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt lookupRecord s� ( c c s� | j t j | d � \ } } | d k r� | r� | j j | d | �V} | d k rz | j | t j t j t j f � q� | j t j | � n t | � d S( NRl t timeoutSeconds( R� R R RD Rk Rm R� R R R� R ( R# R R� RF t doQuery( ( s" /opt/cpanel-ccs/txdav/who/cache.pyRl x s c c s� | j t j | d � \ } } | d k r� | r� | j j | d | �V} | d k rz | j | t j t j t j f � q� | j t j | � n t | � d S( NRn R� ( R� R R RD Rk Ro R� R R R� R ( R# R R� RF R� ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyRn � s c c s� | j t j | j | f d � \ } } | d k r� | r� | j j | | d | �V} | d k r� | j | t j t j t j f � q� | j t j | j | f � n t | � d S( NRp R� ( R� R R R? RD Rk Rq R� R R R� R ( R# RJ R R� RF R� ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyRp � s c c s� | j t j | d � \ } } | d k r� | r� | j j | d | d | �V} t | � d k r� | j t | � d t j t j t j t j f � q� t | � d k r� | j t j | � q� n | g } t | � d S( NRr t limitResultsR� i i ( R� R R RD Rk Rs t lenR� t listR R R R� R ( R# R R� R� RF R� t records( ( s" /opt/cpanel-ccs/txdav/who/cache.pyRr � s" c C s | j j S( N( Rk RJ ( R# ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyRJ � s c C s | j j � S( N( Rk t recordTypes( R# ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� � s c C s( | j j | d | d | d | d | �S( NR� R� R� R� ( Rk t recordsFromExpression( R# t expressionR� R� R� R� ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� � s c C s | j j | | d | d | �S( NR� R� ( Rk t recordsWithFieldValue( R# R; RH R� R� ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� � s c C s | j j | d | �S( Nt create( Rk t updateRecords( R# R� R� ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� � s c C s | j j | � S( N( Rk t removeRecords( R# t uids( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� s c C s | j j | d | d | �S( NR� R� ( Rk t recordsWithRecordType( R# RJ R� R� ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� s c C s" | j j | d | d | d | �S( Nt contextR� R� ( Rk t recordsMatchingTokens( R# t tokensR� R� R� ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� s c C s" | j j | | | d | d | �S( NR� R� ( Rk t recordsMatchingFields( R# R8 t operandRJ R� R� ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� s c C s | j j � S( N( Rk t" recordsWithDirectoryBasedDelegates( R# ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� s c C s t j | | d | �S( NR� ( R t recordWithCalendarUserAddress( R# t cuaR� ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� # s c C s | j j � S( N( Rk t serversDB( R# ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� ) s c c s; | j d k r | j j � n | j � | j j � Vd S( N( R� RD Rh Rx Rk ( R# ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyRh , s c C s | j j � S( N( Rk t stats( R# ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR� 3 s N(( R R R R t BaseFieldNameR R; R^ R( R~ R| Rx R� R� R� R� R� R� R RD Rl Rn Rp Rr t propertyRJ R� R� R� Ri R� R� R� R� R� R� R� R� Rh R� ( ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyR sJ % 2 " w( (/ R t __all__R[ RZ R@ t zope.interfaceR t twistedcaldav.memcacheclientR R t twistedcaldav.configR t twisted.internet.deferR R t twext.python.logR t twext.who.directoryR Rj t twext.who.idirectoryR R R� R RP R t twext.who.utilR t txdav.common.idirectoryserviceR t txdav.dps.clientRL t txdav.who.directoryR t txdav.who.idirectoryt twisted.python.constantsR R R R RQ R R t objectR R ( ( ( s" /opt/cpanel-ccs/txdav/who/cache.pyt <module> s. "" �