Skip to content

Commit 7569cf6

Browse files
Xianyang Liusrowen
authored andcommitted
[SPARK-18420][BUILD] Fix the errors caused by lint check in Java
## What changes were proposed in this pull request? Small fix, fix the errors caused by lint check in Java - Clear unused objects and `UnusedImports`. - Add comments around the method `finalize` of `NioBufferedFileInputStream`to turn off checkstyle. - Cut the line which is longer than 100 characters into two lines. ## How was this patch tested? Travis CI. ``` $ build/mvn -T 4 -q -DskipTests -Pyarn -Phadoop-2.3 -Pkinesis-asl -Phive -Phive-thriftserver install $ dev/lint-java ``` Before: ``` Checkstyle checks failed at following occurrences: [ERROR] src/main/java/org/apache/spark/network/util/TransportConf.java:[21,8] (imports) UnusedImports: Unused import - org.apache.commons.crypto.cipher.CryptoCipherFactory. [ERROR] src/test/java/org/apache/spark/network/sasl/SparkSaslSuite.java:[516,5] (modifier) RedundantModifier: Redundant 'public' modifier. [ERROR] src/main/java/org/apache/spark/io/NioBufferedFileInputStream.java:[133] (coding) NoFinalizer: Avoid using finalizer method. [ERROR] src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeMapData.java:[71] (sizes) LineLength: Line is longer than 100 characters (found 113). [ERROR] src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java:[112] (sizes) LineLength: Line is longer than 100 characters (found 110). [ERROR] src/test/java/org/apache/spark/sql/catalyst/expressions/HiveHasherSuite.java:[31,17] (modifier) ModifierOrder: 'static' modifier out of order with the JLS suggestions. [ERROR]src/main/java/org/apache/spark/examples/ml/JavaLogisticRegressionWithElasticNetExample.java:[64] (sizes) LineLength: Line is longer than 100 characters (found 103). [ERROR] src/main/java/org/apache/spark/examples/ml/JavaInteractionExample.java:[22,8] (imports) UnusedImports: Unused import - org.apache.spark.ml.linalg.Vectors. [ERROR] src/main/java/org/apache/spark/examples/ml/JavaInteractionExample.java:[51] (regexp) RegexpSingleline: No trailing whitespace allowed. ``` After: ``` $ build/mvn -T 4 -q -DskipTests -Pyarn -Phadoop-2.3 -Pkinesis-asl -Phive -Phive-thriftserver install $ dev/lint-java Using `mvn` from path: /home/travis/build/ConeyLiu/spark/build/apache-maven-3.3.9/bin/mvn Checkstyle checks passed. ``` Author: Xianyang Liu <xyliu0530@icloud.com> Closes #15865 from ConeyLiu/master.
1 parent a75e3fe commit 7569cf6

File tree

9 files changed

+25
-9
lines changed

9 files changed

+25
-9
lines changed

common/network-common/src/main/java/org/apache/spark/network/util/TransportConf.java

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@
1818
package org.apache.spark.network.util;
1919

2020
import com.google.common.primitives.Ints;
21-
import org.apache.commons.crypto.cipher.CryptoCipherFactory;
2221

2322
/**
2423
* A central location that tracks all the settings we expose to users.

common/network-common/src/test/java/org/apache/spark/network/sasl/SparkSaslSuite.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -513,7 +513,7 @@ private static class EncryptionCheckerBootstrap extends ChannelOutboundHandlerAd
513513
boolean foundEncryptionHandler;
514514
String encryptHandlerName;
515515

516-
public EncryptionCheckerBootstrap(String encryptHandlerName) {
516+
EncryptionCheckerBootstrap(String encryptHandlerName) {
517517
this.encryptHandlerName = encryptHandlerName;
518518
}
519519

core/src/main/java/org/apache/spark/io/NioBufferedFileInputStream.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,8 +130,10 @@ public synchronized void close() throws IOException {
130130
StorageUtils.dispose(byteBuffer);
131131
}
132132

133+
//checkstyle.off: NoFinalizer
133134
@Override
134135
protected void finalize() throws IOException {
135136
close();
136137
}
138+
//checkstyle.on: NoFinalizer
137139
}

dev/checkstyle.xml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,20 @@
5252
<property name="file" value="dev/checkstyle-suppressions.xml"/>
5353
</module>
5454

55+
<!--
56+
If you wish to turn off checking for a section of code, you can put a comment in the source
57+
before and after the section, with the following syntax:
58+
59+
// checkstyle:off no.XXX (such as checkstyle.off: NoFinalizer)
60+
... // stuff that breaks the styles
61+
// checkstyle:on
62+
-->
63+
<module name="SuppressionCommentFilter">
64+
<property name="offCommentFormat" value="checkstyle.off\: ([\w\|]+)"/>
65+
<property name="onCommentFormat" value="checkstyle.on\: ([\w\|]+)"/>
66+
<property name="checkFormat" value="$1"/>
67+
</module>
68+
5569
<!-- Checks for whitespace -->
5670
<!-- See http://checkstyle.sf.net/config_whitespace.html -->
5771
<module name="FileTabCharacter">
@@ -168,5 +182,6 @@
168182
<module name="UnusedImports"/>
169183
<module name="RedundantImport"/>
170184
<module name="RedundantModifier"/>
185+
<module name="FileContentsHolder"/>
171186
</module>
172187
</module>

examples/src/main/java/org/apache/spark/examples/ml/JavaInteractionExample.java

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@
1919

2020
import org.apache.spark.ml.feature.Interaction;
2121
import org.apache.spark.ml.feature.VectorAssembler;
22-
import org.apache.spark.ml.linalg.Vectors;
2322
import org.apache.spark.sql.*;
2423
import org.apache.spark.sql.types.DataTypes;
2524
import org.apache.spark.sql.types.Metadata;
@@ -48,7 +47,7 @@ public static void main(String[] args) {
4847
RowFactory.create(5, 9, 2, 7, 10, 7, 3),
4948
RowFactory.create(6, 1, 1, 4, 2, 8, 4)
5049
);
51-
50+
5251
StructType schema = new StructType(new StructField[]{
5352
new StructField("id1", DataTypes.IntegerType, false, Metadata.empty()),
5453
new StructField("id2", DataTypes.IntegerType, false, Metadata.empty()),

examples/src/main/java/org/apache/spark/examples/ml/JavaLogisticRegressionWithElasticNetExample.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,8 +60,8 @@ public static void main(String[] args) {
6060
LogisticRegressionModel mlrModel = mlr.fit(training);
6161

6262
// Print the coefficients and intercepts for logistic regression with multinomial family
63-
System.out.println("Multinomial coefficients: "
64-
+ lrModel.coefficientMatrix() + "\nMultinomial intercepts: " + mlrModel.interceptVector());
63+
System.out.println("Multinomial coefficients: " + lrModel.coefficientMatrix()
64+
+ "\nMultinomial intercepts: " + mlrModel.interceptVector());
6565
// $example off$
6666

6767
spark.stop();

sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,8 @@ public void pointTo(Object baseObject, long baseOffset, int sizeInBytes) {
109109
// Read the number of elements from the first 8 bytes.
110110
final long numElements = Platform.getLong(baseObject, baseOffset);
111111
assert numElements >= 0 : "numElements (" + numElements + ") should >= 0";
112-
assert numElements <= Integer.MAX_VALUE : "numElements (" + numElements + ") should <= Integer.MAX_VALUE";
112+
assert numElements <= Integer.MAX_VALUE :
113+
"numElements (" + numElements + ") should <= Integer.MAX_VALUE";
113114

114115
this.numElements = (int)numElements;
115116
this.baseObject = baseObject;

sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeMapData.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,8 @@ public void pointTo(Object baseObject, long baseOffset, int sizeInBytes) {
6868
// Read the numBytes of key array from the first 8 bytes.
6969
final long keyArraySize = Platform.getLong(baseObject, baseOffset);
7070
assert keyArraySize >= 0 : "keyArraySize (" + keyArraySize + ") should >= 0";
71-
assert keyArraySize <= Integer.MAX_VALUE : "keyArraySize (" + keyArraySize + ") should <= Integer.MAX_VALUE";
71+
assert keyArraySize <= Integer.MAX_VALUE :
72+
"keyArraySize (" + keyArraySize + ") should <= Integer.MAX_VALUE";
7273
final int valueArraySize = sizeInBytes - (int)keyArraySize - 8;
7374
assert valueArraySize >= 0 : "valueArraySize (" + valueArraySize + ") should >= 0";
7475

sql/catalyst/src/test/java/org/apache/spark/sql/catalyst/expressions/HiveHasherSuite.java

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,6 @@
2828
import java.util.Set;
2929

3030
public class HiveHasherSuite {
31-
private final static HiveHasher hasher = new HiveHasher();
3231

3332
@Test
3433
public void testKnownIntegerInputs() {

0 commit comments

Comments
 (0)