Question

In: Computer Science

WRITE A JAVA PROGRAM TO IMPLEMENT THE CONCEPT OF INDEX (Create index in text file) full...

WRITE A JAVA PROGRAM TO IMPLEMENT THE CONCEPT OF INDEX (Create index in text file) full code

Solutions

Expert Solution

Create a document

  • Create a method to get a lucene document from a text file.

  • Create various types of fields which are key value pairs containing keys as names and values as contents to be indexed.

  • Set field to be analyzed or not. In our case, only contents is to be analyzed as it can contain data such as a, am, are, an etc. which are not required in search operations.

  • Add the newly created fields to the document object and return it to the caller method.

private Document getDocument(File file) throws IOException {
   Document document = new Document();
   
   //index file contents
   Field contentField = new Field(LuceneConstants.CONTENTS, 
      new FileReader(file));
   
   //index file name
   Field fileNameField = new Field(LuceneConstants.FILE_NAME,
      file.getName(),
      Field.Store.YES,Field.Index.NOT_ANALYZED);
   
   //index file path
   Field filePathField = new Field(LuceneConstants.FILE_PATH,
      file.getCanonicalPath(),
      Field.Store.YES,Field.Index.NOT_ANALYZED);

   document.add(contentField);
   document.add(fileNameField);
   document.add(filePathField);

   return document;
}   

 

Create a IndexWriter

IndexWriter class acts as a core component which creates/updates indexes during indexing process. Follow these steps to create a IndexWriter −

Step 1 − Create object of IndexWriter.

Step 2 − Create a Lucene directory which should point to location where indexes are to be stored.

Step 3 − Initialize the IndexWriter object created with the index directory, a standard analyzer having version information and other required/optional parameters.

private IndexWriter writer;

public Indexer(String indexDirectoryPath) throws IOException {
   //this directory will contain the indexes
   Directory indexDirectory = 
      FSDirectory.open(new File(indexDirectoryPath));
   
   //create the indexer
   writer = new IndexWriter(indexDirectory, 
      new StandardAnalyzer(Version.LUCENE_36),true,
      IndexWriter.MaxFieldLength.UNLIMITED);
}

 

Start Indexing Process

The following program shows how to start an indexing process −

private void indexFile(File file) throws IOException {
   System.out.println("Indexing "+file.getCanonicalPath());
   Document document = getDocument(file);
   writer.addDocument(document);
}
 

Example Application

To test the indexing process, we need to create a Lucene application test.

Step Description
1

Create a project with a name LuceneFirstApplication under a package com.tutorialspoint.lucene as explained in the Lucene - First Application chapter. You can also use the project created in Lucene - First Application chapter as such for this chapter to understand the indexing process.

2

Create LuceneConstants.java,TextFileFilter.java and Indexer.java as explained in the Lucene - First Application chapter. Keep the rest of the files unchanged.

3

Create LuceneTester.java as mentioned below.

4

Clean and build the application to make sure the business logic is working as per the requirements.

LuceneConstants.java

This class is used to provide various constants to be used across the sample application.

package com.test.lucene;

public class LuceneConstants {
   public static final String CONTENTS = "contents";
   public static final String FILE_NAME = "filename";
   public static final String FILE_PATH = "filepath";
   public static final int MAX_SEARCH = 10;
}
 

TextFileFilter.java

This class is used as a .txt file filter.

package com.test.lucene;

import java.io.File;
import java.io.FileFilter;

public class TextFileFilter implements FileFilter {

   @Override
   public boolean accept(File pathname) {
      return pathname.getName().toLowerCase().endsWith(".txt");
   }
}
 

Indexer.java

This class is used to index the raw data so that we can make it searchable using the Lucene library.

package com.test.lucene;

import java.io.File;
import java.io.FileFilter;
import java.io.FileReader;
import java.io.IOException;

import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.util.Version;

public class Indexer {

   private IndexWriter writer;

   public Indexer(String indexDirectoryPath) throws IOException {
      //this directory will contain the indexes
      Directory indexDirectory = 
         FSDirectory.open(new File(indexDirectoryPath));

      //create the indexer
      writer = new IndexWriter(indexDirectory, 
         new StandardAnalyzer(Version.LUCENE_36),true,
         IndexWriter.MaxFieldLength.UNLIMITED);
   }

   public void close() throws CorruptIndexException, IOException {
      writer.close();
   }

   private Document getDocument(File file) throws IOException {
      Document document = new Document();

      //index file contents
      Field contentField = new Field(LuceneConstants.CONTENTS, 
         new FileReader(file));
      
      //index file name
      Field fileNameField = new Field(LuceneConstants.FILE_NAME,
         file.getName(),
         Field.Store.YES,Field.Index.NOT_ANALYZED);
      
      //index file path
      Field filePathField = new Field(LuceneConstants.FILE_PATH,
         file.getCanonicalPath(),
         Field.Store.YES,Field.Index.NOT_ANALYZED);

      document.add(contentField);
      document.add(fileNameField);
      document.add(filePathField);

      return document;
   }   

   private void indexFile(File file) throws IOException {
      System.out.println("Indexing "+file.getCanonicalPath());
      Document document = getDocument(file);
      writer.addDocument(document);
   }

   public int createIndex(String dataDirPath, FileFilter filter) 
      throws IOException {
      //get all files in the data directory
      File[] files = new File(dataDirPath).listFiles();

      for (File file : files) {
         if(!file.isDirectory()
            && !file.isHidden()
            && file.exists()
            && file.canRead()
            && filter.accept(file)
         ){
            indexFile(file);
         }
      }
      return writer.numDocs();
   }
}

 

LuceneTester.java

This class is used to test the indexing capability of the Lucene library.

package com.test.lucene;

import java.io.IOException;

public class LuceneTester {
        
   String indexDir = "E:\\Lucene\\Index";
   String dataDir = "E:\\Lucene\\Data";
   Indexer indexer;
   
   public static void main(String[] args) {
      LuceneTester tester;
      try {
         tester = new LuceneTester();
         tester.createIndex();
      } catch (IOException e) {
         e.printStackTrace();
      } 
   }

   private void createIndex() throws IOException {
      indexer = new Indexer(indexDir);
      int numIndexed;
      long startTime = System.currentTimeMillis();      
      numIndexed = indexer.createIndex(dataDir, new TextFileFilter());
      long endTime = System.currentTimeMillis();
      indexer.close();
      System.out.println(numIndexed+" File indexed, time taken: "
         +(endTime-startTime)+" ms");           
   }
}

Data & Index Directory Creation

We have used 10 text files from record1.txt to record10.txt containing names and other details of the students and put them in the directory E:\Lucene\Data. Test Data. An index directory path should be created as E:\Lucene\Index. After running this program, you can see the list of index files created in that folder.

Running the Program

Once you are done with the creation of the source, the raw data, the data directory and the index directory, you can proceed by compiling and running your program. To do this, keep the LuceneTester.Java file tab active and use either the Run option available in the Eclipse IDE or use Ctrl + F11 to compile and run your LuceneTester application. If your application runs successfully, it will print the following message in Eclipse IDE's console −

Indexing E:\Lucene\Data\record1.txt
Indexing E:\Lucene\Data\record10.txt
Indexing E:\Lucene\Data\record2.txt
Indexing E:\Lucene\Data\record3.txt
Indexing E:\Lucene\Data\record4.txt
Indexing E:\Lucene\Data\record5.txt
Indexing E:\Lucene\Data\record6.txt
Indexing E:\Lucene\Data\record7.txt
Indexing E:\Lucene\Data\record8.txt
Indexing E:\Lucene\Data\record9.txt
10 File indexed, time taken: 109 ms

Once you've run the program successfully, you will have the following content in your index directory −


Related Solutions

WRITE A C++ PROGRAM TO IMPLEMENT THE CONCEPT OF INDEX (Create index in text file)
WRITE A C++ PROGRAM TO IMPLEMENT THE CONCEPT OF INDEX (Create index in text file)
Write a java program: Write a program that creates a text file. Write to the file...
Write a java program: Write a program that creates a text file. Write to the file three lines each line having a person's name. In the same program Append to the file one line of  'Kean University'.  In the same program then Read the file and print the four lines without lines between.
Please write a java program to write to a text file and to read from a...
Please write a java program to write to a text file and to read from a text file.
Write a C++ program to create a text file. Your file should contain the following text:...
Write a C++ program to create a text file. Your file should contain the following text: Batch files are text files created by programmer. The file is written in notepad. Creating a text file and writing to it by using fstream: to write to a file, you need to open thew file as write mode. To do so, include a header filr to your program. Create an object of type fsrteam. Open the file as write mode. Reading from a...
write a program in Java that can take a given text file and then compress it...
write a program in Java that can take a given text file and then compress it with Huffman coding due 20 october 2020 Huffman coding can be used to “zip” a text file to save space. You are required to write a program in Java that can take a given text file and then compress it with Huffman coding. Your program should be able to decompress the zipped files as well [4 marks]. In addition, show percentage gain in data...
Write a program that reads a Java source file and produces an index of all identifiers...
Write a program that reads a Java source file and produces an index of all identifiers in the file. For each identifier, print all lines in which it occurs. For simplicity, we will consider each string consisting only of letters, numbers, and underscores an identifer. Declare a Scanner in for reading from the source file and call in.useDelimiter("[^AZa-z0-9_]+"). Then each call to next returns an identifier. Java. Explain logic used lease.
In JAVA Write a brief program that writes your name to a file in text format...
In JAVA Write a brief program that writes your name to a file in text format and then reads it back. Use the PrintWriter and Scanner classes.
I need to write a java program (in eclipse) that will read my text file and...
I need to write a java program (in eclipse) that will read my text file and replace specific placeholders with information provided in a second text file. For this assignment I am given a text file and I must replace <N>, <A>, <G>, with the information in the second file. For example the information can be John 22 male, and the template will then be modified and saved into a new file or files (because there will be multiple entries...
In Java language. Write a brief program that writes your name to a file in text...
In Java language. Write a brief program that writes your name to a file in text format and then reads it back. Use the PrintWriter and Scanner classes.
Write a JAVA program that reads a text file into RAM efficiently, takes a regular expression...
Write a JAVA program that reads a text file into RAM efficiently, takes a regular expression from the user, and then prints every line that matches the RE.
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT