Activity

  • John Walker posted an update in the group Group logo of UpdatesUpdates 4 months, 3 weeks ago

    2020 July 7

    Starting at 06:08:31 UTC we came under an intense vulnerability
    scan and denial of service attack originating from 65.52.64.35,
    which is within the large 65.52.0.0/14 IP block registered
    directly to:
        OrgName:        Microsoft Corporation
        OrgId:          MSFT
        Address:        One Microsoft Way
        City:           Redmond
        StateProv:      WA
        PostalCode:     98052
        Country:        US
        RegDate:        1998-07-09
    This looks very similar to the attack we experienced on
    2020-06-25, coincidentally (?) starting at around 06:00 UTC.  As
    with that attack, it begins with a litany of probes for
    vulnerabilities coming in at the rate of between two and four
    per second.  This continued through 06:10:16, when it began
    probing user profiles for 13 seconds.  Then, at 06:10:29, like
    the last time, it switched into a denial of service attack,
    hitting the home page, which is particularly costly for the
    server to prepare, at the rate of three or four times per
    second.  By 06:10:43 this had blown up the PHP_FPM child worker
    process memory limit and began to result in 500 errors for many
    requests.  This mindless attack continued until 06:15:04, after
    which we haven't seen this IP address.
    
    The User Agent on these was mostly:
        Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:28.0)
            Gecko/20100101 Firefox/28.0
    which we also saw in the June 25 attack.  When it went into
    denial of service mode, the User Agent switched to:
        Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0)
            Gecko/20100101 Firefox/72.0
    again seen in the last attack.  In the next to last time it hit
    before going away, the User Agent was:
        python-requests/2.24.0
    and there were a few of these mixed in earlier.  We had never
    seen this IP address prior to this attack, and we haven't seen
    it since.
    
    I firewalled the IP address.  It's time to think about
    mitigation of these in Gardol, since we've now seen two of them,
    one from the UK and one from Microsoft, both of which rendered
    the site unusable for several minutes.  It would be easy to
    detect rapid-fire hits on the home page without intervening hits
    elsewhere, but that would catch the attack only after it
    switches to denial of service mode.  I'll have to analyse its
    requests in the initial vulnerability scan mode and see if I can
    nip it in the bud earlier.
    
    I must note that the "Site Health" feature of WordPress 5,
    which send E-mail to administrators when the site sends a 500
    to a user, fails to include the time of the event in the
    E-mail, which is the single most useful thing in analysing
    a failure from the access_log.