Project

General

Profile

Feature #251

Provide support for MapReduce Java Library

Added by Tanvi Marballi over 4 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
High
Target version:
% Done:

100%

Estimated Time:
Deployment type:
AppScale - self deployed

History

#1 Updated by Tanvi Marballi over 4 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 80

Changes:

1) com.google.appengine.tools.mapreduce - MapSettings.java
- Removed the imports for ModulesException, ModulesService and ModulesServiceFactory
- Changed the MODULES_EXCEPTION_HANDLER to call the generic Exception.class
- Changed the method (line 275) toShardedJobSettings to remove all Modules reference. Hard coded the version to “version0” and module to “module”

2) com.google.appengine.tools.mapreduce.bigqueryjobs - BigQueryLoadPollJob.java
- Changed the line 71: DeferredTaskContext.markForRetry() to DeferredTaskContext.setDoNotRetry(false) as sdk 1.8.4 does not have that method, it only has setDoNotRetry().

3) com.google.appengine.tools.mapreduce.impl.shardedjob - ShardedJobSettings.java
- Removed the line 8: import com.google.appengine.api.modules.ModulesServiceFactory
- Changed the line 148: return ModulesServiceFactory.getModulesService().getVersionHostname(module, version); —> return “hostname”;

4) com.google.appengine.tools.mapreduce.inputs - DatastoreShardStrategy.java
- Removed the references to query.getNamespace() in splitQuery method (line 218) as the Datastore query object does not have that method in 1.8.4

5) com.google.appengine.tools.mapreduce.servlets - ShufflerServlet.java
- Removed the hostname logic from the method enqueueCallbackTask (line 227) and hard coded the hostname to “hostname”

#2 Updated by Tanvi Marballi over 4 years ago

Currently working on replacing the appengine mapreduce jar by the appscale specific one on the server side and resolving dependencies.

#3 Updated by Tanvi Marballi over 4 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 80 to 100

Also available in: Atom PDF