SCIENTIFIC-LINUX-DEVEL Archives

August 2016

SCIENTIFIC-LINUX-DEVEL@LISTSERV.FNAL.GOV

Options: Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
Ansgar Hockmann-Stolle <[log in to unmask]>
Reply To:
Ansgar Hockmann-Stolle <[log in to unmask]>
Date:
Wed, 24 Aug 2016 10:23:01 +0200
Content-Type:
multipart/signed
Parts/Attachments:
text/plain (5 kB) , smime.p7s (5 kB)
Am 23.08.2016 um 16:06 schrieb Pat Riehecky:
> On 08/23/2016 08:27 AM, Ansgar Hockmann-Stolle wrote:
>>
>> Am 02.06.2016 um 17:08 schrieb Orion Poplawski:
>>> On 05/26/2016 08:42 AM, Pat Riehecky wrote:
>>>>
>>>> On 05/25/2016 05:06 PM, Orion Poplawski wrote:
>>>>> On 05/19/2016 06:49 PM, Steven Haigh wrote:
>>>>>> Just to bang the drum, seems this has come up again:
>>>>>>
>>>>>> /etc/cron.daily/0yum-daily.cron:
>>>>>>
>>>>>> Not using downloaded repomd.xml because it is older than what we
>>>>>> have:
>>>>>>     Current   : Wed May 18 06:06:41 2016
>>>>>>     Downloaded: Wed May 18 06:06:39 2016
>>>>>>
>>>>>> Sadly the cron job doesn't tell me which repo this was on.
>>>>>>
>>>>> It's appeared here again today with the sl7 fastbugs repo.
>>>>>
>>>>> I've attached the two different files.
>>>>>
>>>> :(
>>>>
>>>> I've updated the push process yet again.  At every step of the
>>>> process it
>>>> verifies the timestamp is current.
>>>>
>>>> In theory this will really really really fix this.
>>> Note that my original diagnosis of needing to set --revision I
>>> believe now was
>>> incorrect and that the timestamp of the repo is determined by the newest
>>> metadata file listed in repomd.xml.
>> As the problem still persists I tried to understand what happens by
>> looking at the source of yum ...
>>
>> In yum/repoMDObject.py the timestamp of one data element of repomd.xml
>> will be the input of "int()":
>>
>>      try:
>>          nts = int(thisdata.timestamp)
>>          if nts > self.timestamp: # max() not in old python
>>              self.timestamp = nts
>>          except:
>>              pass
>>
>> Attached you will find the current repomd.xml from:
>>
>> ftp://ftp.scientificlinux.org/linux/scientific/7/x86_64/updates/security/repodata/repomd.xml
>>
>>
>> as "ftp_repomd.xml" and the cached one from one of our SL7 machines as
>> "cached_repomd.xml". They are different in indenting and sequence! But
>> the main problem is the bad timestamp of the updateinfo data element in
>> ftp_repomd.xml:
>>
>> <data type="updateinfo">
>> [...]
>>    <timestamp>1471637392.18</timestamp>
>>
>> This is not an int. It will be ignored without a warning and the
>> resulting timestamp is not raised. The timestamp of the last good data
>> element in the repomd.xml will be used. That is the reason for our
>> warning with so small time differences.
>>
>> Please find the tool which creates the bad timestamp ;-)
>>
>> Ciao
>>     Ansgar Hockmann-Stolle
>>
> 
> Thanks for the report!  I'll confess a bit of sorrow, but that is mostly
> because I'd swear I'd fixed this a dozen or so times now.
> 
> At this point, the thing that writes the updateinfo.xml makes sure it is
> older than repomd.xml, the thing that adds it into repomd.xml makes sure
> it is older than repomd.xml before it was added, and the report script
> checks all of this.
> 
> Ok, I'd swear now that every piece of the process tries to set the
> timestamp right at every step... of course I've said that before...

Do we talk about the same timestamp? I just did this:

> [root@vm476 ~]# yum clean metadata
> Quellen werden aufgeräumt:rz sl sl-extras sl-fastbugs sl-security
> 21 metadata Dateien entfernt
> 9 sqlite Dateien entfernt
> 0 metadata Dateien entfernt
> [root@vm476 ~]# yum check-update
> rz                                                       |  951 B     00:00     
> sl                                                       | 3.7 kB     00:00     
> sl-extras                                                | 3.0 kB     00:00
> sl-fastbugs                                              | 2.9 kB     00:00
> sl-security                                              | 2.9 kB     00:00
> (1/9): sl-extras/x86_64/updateinfo                         |  23 kB   00:00
> (2/9): sl/x86_64/group_gz                                  | 107 kB   00:00
> (3/9): sl-fastbugs/x86_64/updateinfo                       |  76 kB   00:00
> (4/9): sl-extras/x86_64/primary_db                         | 118 kB   00:00
> (5/9): sl-security/x86_64/updateinfo                       |  82 kB   00:00
> (6/9): sl/x86_64/updateinfo                                | 1.4 MB   00:02
> (7/9): sl-fastbugs/x86_64/primary_db                       | 1.6 MB   00:02
> (8/9): sl-security/x86_64/primary_db                       | 2.6 MB   00:02
> (9/9): sl/x86_64/primary_db                                | 4.6 MB   00:03
> rz/7/x86_64/primary                                        |  13 kB   00:00
> rz                                                                        33/33
> 
> kernel.x86_64                        3.10.0-327.28.3.el7             sl-security
> kernel-tools.x86_64                  3.10.0-327.28.3.el7             sl-security
> kernel-tools-libs.x86_64             3.10.0-327.28.3.el7             sl-security
> [root@vm476 ~]# egrep 'timestamp.*\.' $(find /var/cache/yum/  -name repomd.xml)
> /var/cache/yum/x86_64/7/sl-fastbugs/repomd.xml:  <timestamp>1471960896.16</timestamp>
> /var/cache/yum/x86_64/7/sl-security/repomd.xml:  <timestamp>1471961274.74</timestamp>

Upps! Yum uses this <timestamp> XML-element, not the inode timestamp.
And with the "." it will fail.

Good luck with bug hunting!

Ansgar

-- 
Ansgar Hockmann-Stolle · Universität Osnabrück · Rechenzentrum
Albrechtstraße 28, 49076 Osnabrück, Deutschland · Raum 31/E77B
+49 541 969-2749 (fax -2470) · http://www.home.uos.de/anshockm



ATOM RSS1 RSS2