On 2016-04-15 02:45, Pat Riehecky wrote: > On 04/14/2016 11:32 AM, Orion Poplawski wrote: >> On 04/14/2016 08:54 AM, Pat Riehecky wrote: >>> On 04/14/2016 09:35 AM, Orion Poplawski wrote: >>>> On 03/30/2016 10:44 AM, Pat Riehecky wrote: >>>>> On 03/30/2016 11:39 AM, Orion Poplawski wrote: >>>>>> On 03/29/2016 08:55 AM, Orion Poplawski wrote: >>>>>>> I get messages like this fairly frequently on our SL7 servers: >>>>>>> >>>>>>> /etc/cron.daily/0yum-daily.cron: >>>>>>> >>>>>>> Not using downloaded repomd.xml because it is older than what we >>>>>>> have: >>>>>>> Current : Fri Mar 25 08:54:39 2016 >>>>>>> Downloaded: Fri Mar 25 08:54:20 2016 >>>>>>> >>>>>>> Today, this is from the sl-security repo: >>>>>>> >>>>>>> -rw-r--r--. 1 root root 3059 Mar 25 08:54 >>>>>>> /var/cache/yum/x86_64/7/sl-security/repomd.xml >>>>>>> >>>>>>> We use a mirroring scheme that is based on time-stamps which may >>>>>>> be >>>>>>> aggravating this locally, and are mirroring from >>>>>>> http://ftp1.scientificlinux.org/linux/scientific >>>>>>> >>>>>>> However it seems that the time-stamps of these files should never >>>>>>> go >>>>>>> backwards. Is there something going on in the SL staging process >>>>>>> causing >>>>>>> this? >>>>>> A little more information. Apparently the <revision> tag in the >>>>>> repomd.xml is >>>>>> treated as the timestamp of the file. Currently for the currently >>>>>> downloaded >>>>>> sl-security the revision is: >>>>>> >>>>>> $ grep revision /var/cache/yum/x86_64/7/sl-security/repomd.xml >>>>>> <revision>1458917662</revision> >>>>>> $ ls -l /var/cache/yum/x86_64/7/sl-security/repomd.xml >>>>>> -rw-r--r--. 1 root root 3059 Mar 25 08:54 >>>>>> /var/cache/yum/x86_64/7/sl-security/repomd.xml >>>>>> $ date --date=@1458917662 >>>>>> Fri Mar 25 08:54:22 MDT 2016 >>>>>> $ stat --printf '%y\n' >>>>>> /var/cache/yum/x86_64/7/sl-security/repomd.xml >>>>>> 2016-03-25 08:54:40.000000000 -0600 >>>>>> >>>>>> Now I don't understand how the cached copy ended up with a >>>>>> slightly newer >>>>>> timestamp - possibly due to some munging by yum after download. >>>>>> >>>>>> For the current version on my mirror: >>>>>> >>>>>> # ls -l 7x/x86_64/updates/security/repodata/repomd.xml >>>>>> -rw-r--r--. 1 apache apache 2965 Mar 29 07:46 >>>>>> 7x/x86_64/updates/security/repodata/repomd.xml >>>>>> # grep revision 7x/x86_64/updates/security/repodata/repomd.xml >>>>>> <revision>1458917662</revision> >>>>>> >>>>>> So despite being newer and different file, it has the same >>>>>> "revision" as the >>>>>> older one. >>>>>> >>>>>> Perhaps you are manually setting --revision in createrepo? By >>>>>> default it >>>>>> sets >>>>>> revision to the unix timestamp at the start of the run. >>>>>> >>>>> I'll alter the publication tools to add this value >>>>> >>>>> Pat >>>> Seems to have happened again: >>>> >>>> # /etc/cron.daily/0yum-daily.cron >>>> Not using downloaded repomd.xml because it is older than what we >>>> have: >>>> Current : Wed Apr 13 09:59:29 2016 >>>> Downloaded: Wed Apr 13 09:59:28 2016 >>>> >>>> -rw-r--r--. 1 root root 3059 Apr 13 09:59 >>>> /var/cache/yum/x86_64/7/sl-security/repomd.xml >>>> >>> http://ftp.scientificlinux.org/linux/scientific/7x/x86_64/updates/security/repodata/repomd.xml >>> >>> >>> Contains the revision field you requested. >>> >>> Data from the publication filesystem: >>> $ pwd >>> /linux/scientific/7x/x86_64/updates/security/repodata >>> $ stat repomd.xml >>> File: ‘repomd.xml’ >>> Size: 2965 Blocks: 64 IO Block: 32768 regular >>> file >>> Device: 2ah/42d Inode: 905695246 Links: 1 >>> Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) >>> Context: system_u:object_r:nfs_t:s0 >>> Access: 2016-04-14 07:01:06.288136000 -0500 >>> Modify: 2016-04-13 11:06:36.036051000 -0500 >>> Change: 2016-04-13 11:06:36.036051000 -0500 >>> Birth: - >>> $ >>> >>> Pat >> Okay, it appears that I misread some of the code originally. It >> appears that >> the "timestamp" of the repomd file is the latest timestamp of all of >> the >> component data entries. >> >> I've attached the old and new repomd.xml files that I have. Both have >> the >> same revision, but the timestamps of the data files are different. >> The order >> and structure of the files are different as well so something >> reprocessed them. >> >> The latest timestamp in the old file is 1460563169 from updateinfo. >> The >> latest timestamp in the new file is 1460563168 from primary_db. >> updateinto is >> not in the old repomd.xml file. >> >> > Hmmmmm..... > > We use modifyrepo to inject the updateinfo data based on the packages > within the metadata. > > I'll see about adjusting the process there. Is there any reason modifyrepo is used instead of just createrepo? I understand it may take longer to do a full (re)create, but I've never run into this problem when I've done repos. My workflow is rather simple, but I do: #!/bin/bash if [ -z $1 ]; then echo "Usage: $0 <reponame>" exit 1 fi createrepo --outputdir=/path/to/repo/$1/x86_64/ /path/to/repo/$1/x86_64/ rsync --delete-after --progress -a /path/to/$1/ livesite:/path/to/repo/$1/ exit 0 You could optionally use the --update parameter to update an existing repo - which may be faster. -- Steven Haigh Email: [log in to unmask] Web: https://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897