Friday, September 21, 2007

NFS Ashburn Farm Servers Details Discussed

Early this month I wrote an article "NFS Fireside I Would Love to Attend" and I was excited to read on the LDS-WARD-CONSULTANT mailing list that Jackson and Sylvia Hott Sonneborn attended this fireside in Virginia. They have done an excellent job writing up their notes from the fireside, and I am reposting them here for you to read.

Ashburn Fireside 19 Sep 2007
Regarding the Ashburn Farm Server
Presiding: Stake President Lynn Chapman, Agent Stake President for the Ashburn Server Farm

We attended the fireside in Ashburn, VA, where the East Coast server farm has been completed for New Family Search (one of 2 in the world at the present). The Ashburn spot was chosen because the area is filled with Internet servers and good security. It takes about 5 levels of security (palm prints) to get through to the cage that houses the thousand servers, hardware and software in Ashburn. It is a small space with lots of electronic equipment, and it generates a lot of heat. The racks are in columns of 40 metal boxes or servers stacked on shelves. It was set up by volunteers. The member volunteers in the 4 surrounding stakes are also manning it round the clock. What dedication from the Saints from that area! Once it is fully operational (testing all operating systems and working out bugs), it will be dedicated so that they can bring the Spirit of the Lord there to protect what they feel is the closest we will get to the Book of Life.

It is amazing to learn of the technological advances in family history. Bro. Greg Steffen, who is in charge of the project, outlined them. He also discussed 3 challenges:
  • First, the amount of research and temple work. From Adam to today, there have been 50-100 billion humans, most living in the last few centuries. Since there was no coordination between researchers, there is massive duplication of records.
  • Second, the project must solve records management. 60 billion records exist globally. 3 billion are in the Granite Vault on 2.4 million microfilm rolls. 70 million new records are generated annually. There are 12 billion individual names on film. It will take 18,059,517, 568, 922, 420 petabytes to house all that information. The records must be widely available, indexed, and searchable.
  • The third problem is records persistence. They have to last forever. Old microfilm lasted 75 years. New microfilm lasts 200 years. Electronic storage lasts ??? maybe less than 10 years. We need a place to store things. We are still microfilming and digitizing records. At present, The Church's collection is five times the size of that in the National Archives. So where do we keep 3 billion records? The Granite Vault is the answer and the servers.
Next Bro. Steffen discussed the evolution of family history from the 1432 Gutenberg Bible to Family Search Indexing. IBM built the first computer in 1944. Now we are adapting the information to the computer. It took 7 years to complete the 1880 census. Now it takes 2 1/2 years to complete a census.

He next discussed Record Imaging and Distribution. We have faster scanners. The raw images get processed into individual compressed images on tapes. When opened, they will enlarge. They will have high and low resolution. The storage site is the Internet, which is available to everybody - the entire world. We will log into the servers here (Ashburn Server Farm), where the data is stored.

Ed Donakey (sp), who is in charge of all strategic relationships to family history, joined Bro. Steffen at the podium. He said that St Louis, Orlando, Billings, and Reno are now online. We did learn that the bigger temples will be the last for the rollout, so our temple (WASHI) will not get on NFS for a while, but one of the speakers said that the goal is to have all temple "rollouting" done by July 2008.

The Church used 200 software developers to develop NFS. At the present time we have 17% duplication in temple work. The New Family Search (NFS) should cut this down to 1 %. Five files have been combined in NFS. I know Ancestral File, IGI, and Pedigree Resource File. Does anyone else know the other 2? Its advantage will be that the temple work will be recorded immediately, and the project will allow for collaboration by various researchers. At the present time there are 17 steps to take a name to the temple. NFS will cut the steps down to two. Members will be able to clear names at home for temple work. We will no longer use floppy disks but will prepare our records on home or FHC computers and print out the record to take to the temple to have cards printed at the temple.

At the present time there are 70,000 people indexing, and that is being cut down so that it is more manageable. They are indexing faster than the servers can handle. The extraction program took 19 years to do the 1880 census. With the volunteers, the 1920 census took 9 months. We will not need PAF as we use it today, but we will still need Family History Centers. Members will have an opportunity to correct the records online. As The Church has attempted to remove the 81 million duplications, they have merged like-records but retain any additional information that exists on all records, so that nothing is lost. Identical information registers on one record.

The Church won the bid to digitize the records in Italy. It needs senior members to serve missions to assist in that endeavor. The Church is also working with governments, societies, and other organizations to digitize their records. For the copyrighted records in the Granite Vault, about 87% of the authors have agreed to let The Church digitize the records.

Bro Steffen views his job as a testimony-strengthening experience to work with such dedicated Saints on the East Coast, and we should never feel bad that we are not in Utah. The Spirit is so strong here with the Saints who worked with him. Many common men (not techies but extraordinary, spiritual men) rose to the occasion and provided invaluable leadership and service for the project.

Bro. Steffen said that he feels this server farm is "the book" spoken of in the Doctrine and Covenants 128:24 that is to be completed for the Lord. He said that the Lord's finger prints are all over this project. This is the largest server farm with a smaller one in Utah and will handle all the NFS records. How blessed we are to be witnesses to this great event, which goes unnoticed by the masses.

(We hope this is fairly accurate. It is difficult to catch all the information when so much is presented.)

Jackson and Sylvia Hott Sonneborn
York, PA


James W. Anderson said...

I heard last week they want even more indexers and are actually scaling up as more people sign on.

As of last Friday, they have 86,000 indexers, with 36,000 of them doing something on any given day. They have 200 to 400 new people a day signing on to do indexing, and while that is down from August when they had as many as 1,200 a day sign on after the Ensign article came out, it is still more than they had before the article. It is likely, then, that the 100,000 goal for the number of indexers by the end of the year will be met by around October General Conference.

They hope to have 500,000 indexers within three years, and eventually, over a million doing indexing. So I think the term 'cutback' may have been a misunderstanding, it may have to do with something else.

layjent said...

A couple items of correction

1) the 18,...,...,... number is bytes, which is equal to 18 petabytes

2) I believe they did not mention "cut back" on the indexers, but they were slow to accept new ones as of right now because there is a slight bottleneck on the church side (indexers are doing great work). However, the impression was that that was being worked on and hopefully soon would not be a problem (I would imagine that if they did turn away people it didn't last long).

Anonymous said...

We will still need PAF at home. PAF is a record we can keep more accurate for OUR OWN USE of our Ancestors. We cannot correct errors on new FamilySearch if we did not contribute. We can dispute, or if able to contact other contributers, may correlate with them to correct some errors.
Because nFS is from so many different sources it is very difficult to get all errors corrected.

Remember PAF is for our own ancestors.
nFS is for the whole world.

Edith Neville

Anonymous said...

Is PAF going away?

Is Personal Ancestral File being discontinued?
What is the future of PAF?
Is PAF compatible with the new FamilySearch?
Does the new FamilySearch replace PAF?
Is PAF dead?
Is PAF being dropped?


PAF Is Still Needed
PAF is one of the genealogy database programs that can produce GEDCOM files, which can be uploaded to the new FamilySearch.

While there are no plans to further develop PAF, it remains a dependable and easy-to-use program. Users of PAF can receive support by telephone or e-mail, as well as through the knowledge base in the Product Support area of Local support may also be available at a family history center or from a family history consultant. There are also inexpensive utility programs that provide enhancements. These include PAF Companion, PAF Insight, and PAF Wiz.

If a PAF user later decides to switch to a different program, his or her data can be exported as a GEDCOM file and then imported to any commercial genealogy database program that uses GEDCOM. Some of these programs have the ability to import a PAF file directly so the data does not have to be reinput, but simply loaded into the new program.

PAF and the New FamilySearch Are Complementary Products
PAF stores a wide range of data with sources and voluminous notes. With it, users can perform advanced searches of data and can print a variety of reports, including books. PAF has a built-in "Print-to-file" (RTF) feature, which allows users to create electronic copies of reports and charts that can be sent as e-mail attachments. If a free PDF writer such as PrimoPDF or CutePDF is installed on the computer, PAF can use it to create PDF copies of reports and charts that could then be e-mailed or even posted to a Web site. PAF has a Preview feature that allows users to see a report before they print it. Users can also link multimedia files to their PAF data. Many of the personal genealogical databases on the Internet were created using PAF.

The New FamilySearch Will Play a Different Role
The new FamilySearch will replace TempleReady. You will be able to prepare names using the new FamilySearch and then take them directly to any temple (you will not have to take them to a family history center anymore). The new FamilySearch will make it easier for you to work with others on ancestral family lines since you can all access the same information. You will be able to see where individuals fit in the context of their whole family, unlike the IGI, which shows only individual births, marriages, and deaths. The new FamilySearch will also allow you to challenge errors that have been made and work to correct them. As the program continues to be developed, the role of the new FamilySearch will certainly be increased.