Subject: | |
From: | |
Reply To: | |
Date: | Fri, 15 Apr 2016 11:20:17 +1000 |
Content-Type: | text/plain |
Parts/Attachments: |
|
|
On 2016-04-15 02:45, Pat Riehecky wrote:
> On 04/14/2016 11:32 AM, Orion Poplawski wrote:
>> On 04/14/2016 08:54 AM, Pat Riehecky wrote:
>>> On 04/14/2016 09:35 AM, Orion Poplawski wrote:
>>>> On 03/30/2016 10:44 AM, Pat Riehecky wrote:
>>>>> On 03/30/2016 11:39 AM, Orion Poplawski wrote:
>>>>>> On 03/29/2016 08:55 AM, Orion Poplawski wrote:
>>>>>>> I get messages like this fairly frequently on our SL7 servers:
>>>>>>>
>>>>>>> /etc/cron.daily/0yum-daily.cron:
>>>>>>>
>>>>>>> Not using downloaded repomd.xml because it is older than what we
>>>>>>> have:
>>>>>>> Current : Fri Mar 25 08:54:39 2016
>>>>>>> Downloaded: Fri Mar 25 08:54:20 2016
>>>>>>>
>>>>>>> Today, this is from the sl-security repo:
>>>>>>>
>>>>>>> -rw-r--r--. 1 root root 3059 Mar 25 08:54
>>>>>>> /var/cache/yum/x86_64/7/sl-security/repomd.xml
>>>>>>>
>>>>>>> We use a mirroring scheme that is based on time-stamps which may
>>>>>>> be
>>>>>>> aggravating this locally, and are mirroring from
>>>>>>> http://ftp1.scientificlinux.org/linux/scientific
>>>>>>>
>>>>>>> However it seems that the time-stamps of these files should never
>>>>>>> go
>>>>>>> backwards. Is there something going on in the SL staging process
>>>>>>> causing
>>>>>>> this?
>>>>>> A little more information. Apparently the <revision> tag in the
>>>>>> repomd.xml is
>>>>>> treated as the timestamp of the file. Currently for the currently
>>>>>> downloaded
>>>>>> sl-security the revision is:
>>>>>>
>>>>>> $ grep revision /var/cache/yum/x86_64/7/sl-security/repomd.xml
>>>>>> <revision>1458917662</revision>
>>>>>> $ ls -l /var/cache/yum/x86_64/7/sl-security/repomd.xml
>>>>>> -rw-r--r--. 1 root root 3059 Mar 25 08:54
>>>>>> /var/cache/yum/x86_64/7/sl-security/repomd.xml
>>>>>> $ date --date=@1458917662
>>>>>> Fri Mar 25 08:54:22 MDT 2016
>>>>>> $ stat --printf '%y\n'
>>>>>> /var/cache/yum/x86_64/7/sl-security/repomd.xml
>>>>>> 2016-03-25 08:54:40.000000000 -0600
>>>>>>
>>>>>> Now I don't understand how the cached copy ended up with a
>>>>>> slightly newer
>>>>>> timestamp - possibly due to some munging by yum after download.
>>>>>>
>>>>>> For the current version on my mirror:
>>>>>>
>>>>>> # ls -l 7x/x86_64/updates/security/repodata/repomd.xml
>>>>>> -rw-r--r--. 1 apache apache 2965 Mar 29 07:46
>>>>>> 7x/x86_64/updates/security/repodata/repomd.xml
>>>>>> # grep revision 7x/x86_64/updates/security/repodata/repomd.xml
>>>>>> <revision>1458917662</revision>
>>>>>>
>>>>>> So despite being newer and different file, it has the same
>>>>>> "revision" as the
>>>>>> older one.
>>>>>>
>>>>>> Perhaps you are manually setting --revision in createrepo? By
>>>>>> default it
>>>>>> sets
>>>>>> revision to the unix timestamp at the start of the run.
>>>>>>
>>>>> I'll alter the publication tools to add this value
>>>>>
>>>>> Pat
>>>> Seems to have happened again:
>>>>
>>>> # /etc/cron.daily/0yum-daily.cron
>>>> Not using downloaded repomd.xml because it is older than what we
>>>> have:
>>>> Current : Wed Apr 13 09:59:29 2016
>>>> Downloaded: Wed Apr 13 09:59:28 2016
>>>>
>>>> -rw-r--r--. 1 root root 3059 Apr 13 09:59
>>>> /var/cache/yum/x86_64/7/sl-security/repomd.xml
>>>>
>>> http://ftp.scientificlinux.org/linux/scientific/7x/x86_64/updates/security/repodata/repomd.xml
>>>
>>>
>>> Contains the revision field you requested.
>>>
>>> Data from the publication filesystem:
>>> $ pwd
>>> /linux/scientific/7x/x86_64/updates/security/repodata
>>> $ stat repomd.xml
>>> File: ‘repomd.xml’
>>> Size: 2965 Blocks: 64 IO Block: 32768 regular
>>> file
>>> Device: 2ah/42d Inode: 905695246 Links: 1
>>> Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
>>> Context: system_u:object_r:nfs_t:s0
>>> Access: 2016-04-14 07:01:06.288136000 -0500
>>> Modify: 2016-04-13 11:06:36.036051000 -0500
>>> Change: 2016-04-13 11:06:36.036051000 -0500
>>> Birth: -
>>> $
>>>
>>> Pat
>> Okay, it appears that I misread some of the code originally. It
>> appears that
>> the "timestamp" of the repomd file is the latest timestamp of all of
>> the
>> component data entries.
>>
>> I've attached the old and new repomd.xml files that I have. Both have
>> the
>> same revision, but the timestamps of the data files are different.
>> The order
>> and structure of the files are different as well so something
>> reprocessed them.
>>
>> The latest timestamp in the old file is 1460563169 from updateinfo.
>> The
>> latest timestamp in the new file is 1460563168 from primary_db.
>> updateinto is
>> not in the old repomd.xml file.
>>
>>
> Hmmmmm.....
>
> We use modifyrepo to inject the updateinfo data based on the packages
> within the metadata.
>
> I'll see about adjusting the process there.
Is there any reason modifyrepo is used instead of just createrepo? I
understand it may take longer to do a full (re)create, but I've never
run into this problem when I've done repos.
My workflow is rather simple, but I do:
#!/bin/bash
if [ -z $1 ]; then
echo "Usage: $0 <reponame>"
exit 1
fi
createrepo --outputdir=/path/to/repo/$1/x86_64/ /path/to/repo/$1/x86_64/
rsync --delete-after --progress -a /path/to/$1/
livesite:/path/to/repo/$1/
exit 0
You could optionally use the --update parameter to update an existing
repo - which may be faster.
--
Steven Haigh
Email: [log in to unmask]
Web: https://www.crc.id.au
Phone: (03) 9001 6090 - 0412 935 897
|
|
|